forked from DonJayamanne/pythonVSCode
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Components Test Management
Karthik Nadig edited this page Jan 27, 2021
·
1 revision
The extension provides testing for Python code through one of the supported frameworks. This involves a variety of capabilities in addition to just running tests. The following frameworks are supported:
- install selected test framework if not installed
- test discovery
- run all/selected tests & suites
- debug all/selected tests & suites
- breakpoints while running tests
- show (high level) test results
Note that the active Python interpreter is used for all testing operations.
- (somewhat) granular test framework config settings
- dedicated output log
- "Python: Discover Unit Tests"
- "Python: Run All Unit Tests"
- "Python: Run Unit Test Method..."
- "Python: View Unit Test Output"
- status bar element
- shows test results
- triggers quick pick for testing-related operations
- pop-up (& quick pick) to configure test framework
- code lens on tests and suites:
- run/debug
- results
- output panel: "Python Test Log"
- TBD: Test Explorer activity (top-level VSC element)
- "python.unitTest.cwd"
- "python.unitTest.debugPort"
- "python.unitTest.nosetestArgs"
- "python.unitTest.nosetestsEnabled"
- "python.unitTest.nosetestPath"
- "python.unitTest.promptToConfigure"
- "python.unitTest.pyTestArgs"
- "python.unitTest.pyTestEnabled"
- "python.unitTest.pyTestPath"
- "python.unitTest.unittestArgs"
- "python.unitTest.unittestEnabled"
- "python.unitTest.autoTestDiscoverOnSaveEnabled"
All test-related code is contained entirely under a single directory, src/client/unittests/:
src/client/unittest
├── codeLenses
├── common
│ ├── managers
│ ├── services
│ └── testVisitors
├── display
├── nosetest
│ └── services
├── pytest
│ └── services
└── unittest
└── services
files: external API
src/client/unittest
├── serviceRegistry.ts
└── types.ts
files: top-level code
src/client/unittest
├── common
│ ├── argumentsHelper.ts
│ ├── constants.ts
│ ├── debugLauncher.ts
│ ├── managers
│ │ ├── baseTestManager.ts
│ │ └── testConfigurationManager.ts
│ ├── runner.ts
│ ├── services
│ │ ├── configSettingService.ts
│ │ ├── storageService.ts
│ │ ├── testManagerService.ts
│ │ ├── testResultsService.ts
│ │ ├── unitTestDiagnosticService.ts
│ │ └── workspaceTestManagerService.ts
│ ├── testUtils.ts
│ ├── testVisitors
│ │ ├── flatteningVisitor.ts
│ │ ├── folderGenerationVisitor.ts
│ │ └── resultResetVisitor.ts
│ ├── types.ts
│ └── xUnitParser.ts
└── main.ts
files: configuration:
src/client/unittest
├── configurationFactory.ts
├── configuration.ts
├── nosetest
│ └── testConfigurationManager.ts
├── pytest
│ └── testConfigurationManager.ts
└── unittest
└── testConfigurationManager.ts
files: testing frameworks:
src/client/unittest
├── nosetest
│ ├── main.ts
│ ├── runner.ts
│ └── services
│ ├── argsService.ts
│ ├── discoveryService.ts
│ └── parserService.ts
├── pytest
│ ├── main.ts
│ ├── runner.ts
│ └── services
│ ├── argsService.ts
│ ├── discoveryService.ts
│ ├── parserService.ts
│ └── testMessageService.ts
└── unittest
├── helper.ts
├── main.ts
├── runner.ts
├── services
│ ├── argsService.ts
│ ├── discoveryService.ts
│ └── parserService.ts
└── socketServer.ts
files: UI features
src/client/unittest
├── codeLenses
│ ├── main.ts
│ └── testFiles.ts
└── display
├── main.ts
└── picker.ts
- config
- test discovery
- code lens
- picker
- runner
- source:
pythonFiles/testing_tools/adapter/__main__.py - invocation:
[PYTHON] [SCRIPT] discover pytest [ARGS] -- [PYTEST ARGS]- SCRIPT (any python):
pythonFiles/testing_tools/run_adapter.py - SCRIPT (python3-only):
-m pythonFiles.testing_tools.adapter
- args:
-
--simple- simplified output for use while debugging -
--no-hide-stdio- do not hide pytest stdout/stderr during discover (useful when debugging) -
--pretty- format output for readability
-
- implicit pytest args:
--collect-only
OUTPUT: ROOTED_TESTS[]
ROOTED_TESTS: {
# Uniquely identifies test root.
"rootid": str
# Absolute path to test root.
"root": str
# The discovered files, suites, etc. under this root.
"parents": PARENT[]
# The discovered tests under this root.
"tests": TEST[]
}
PARENT: {
# Uniquely identifies the parent.
"id": str
# The kind of parent.
"kind": "folder"|"file"|"suite"|"function"|"subtest"
# A human-friendly identifier.
"name": str
# The parent's parent (i.e. in a tree from the root.
"parentid": str
}
TEST: {
# Uniquely identifies the test.
"id": str
# A human-friendly identifier.
"name": str
# The location of the test's source code.
"source": str ("<FILENAME>:<LINENO>")
# Any supported markers associated with the test.
"markers": ("skip"|"skip-if"|"expected-failure")[]
# The ID of the test's parent.
"parentid": str
}
Notes:
- IDs may be used to identify a test or parent to the test framework (e.g. to run specific tests)
- every child of the root will have the "rootid" as its "parentid"
- supported test markers are framework-agnostic but correspond to framework-specific ones
- a test is considered a "subtest" if its parent is a function or a subtest
- a subtest is a test that is effectively a parameterization of a more general test
- in pytest parameterized tests are the only kind of subtest
- a subtest may be a parent if there is a nested subtest in it
- this is not supported in pytest, but is in unittest
Example:
[{
"rootid": ".",
"root": "/x/y/z",
"parents": [{
"id": "./test_spam.py",
"kind": "file",
"name": "test_spam.py",
"parentid": "."
}, {
"id": "./test_spam.py::SpamTests",
"kind": "suite",
"name": "SpamTests",
"parentid": "./test_spam.py"
},
"tests" [{
"id": "./test_spam.py::test_all",
"name": "test_all",
"source": "test_spam.py:11",
"markers": ["skip", "expected-failure"],
"parentid": "./test_spam.py"
}, {
"id": "./test_spam.py::SpamTests::test_spam1",
"name": "test_spam1",
"source": "test_spam.py:23",
"markers": ["skip"],
"parentid": "./test_spam.py::SpamTests"
}]
}]