Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Conversation

marcphilipp
Copy link
Member

@marcphilipp marcphilipp commented Feb 20, 2023

Overview

  • Add command line option for listing all tests
  • Add getter for test identifiers in test plan
  • List all tests by skipping them
  • Add & implement details printing listener interface
  • Omit the arrow symbol when listing tests
  • List all tests
  • Fix indentation
  • Add API annotation
  • Fix hierarchy when listing tests in tree mode
  • Omit icon when blank
  • Add integration test
  • Make TestPlan iterable
  • Remove unnecessary Supplier
  • Add test and fix flat mode
  • Add test and fix verbose mode
  • Use Visitor instead of Iterator
  • Add summary of found containers/tests
  • Add tests for summary and none detail modes

I hereby agree to the terms of the JUnit Contributor License Agreement.


Definition of Done

@marcphilipp marcphilipp self-assigned this Feb 20, 2023
@marcphilipp marcphilipp linked an issue Feb 20, 2023 that may be closed by this pull request
@marcphilipp marcphilipp force-pushed the marc/2854-console-launcher-discover branch from 3258b66 to f5f0da3 Compare February 21, 2023 18:27
@mpkorstanje
Copy link
Contributor

How shell script friendly is this command? Can do something like:

junit --list-tests > tests.txt
cat tests.txt | junit 

@andrii-lemdianov
Copy link

andrii-lemdianov commented Mar 10, 2023

Dear @marcphilipp,

Thank You very much for implementing this feature. It is amazing.

I tried it a bit and found mismatches between the results of test discovery and test execution for the parameterized tests: 1 test after discovery and 4 tests after execution.

It could be done by design.

JDK:

java -version
openjdk version "17.0.6" 2023-01-17 LTS
OpenJDK Runtime Environment Corretto-17.0.6.10.1 (build 17.0.6+10-LTS)
OpenJDK 64-Bit Server VM Corretto-17.0.6.10.1 (build 17.0.6+10-LTS, mixed mode, sharing)

Test code:

import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
class MavenExampleTest {
    @Test
    void shouldRun() {}

    @ParameterizedTest
    @ValueSource(strings = { "racecar", "radar", "able was I ere I saw elba" })
    void palindromes(String candidate){}
}

Test discovery:

java -jar junit-platform-console-standalone-1.10.0-20230310.021933-1.jar --class-path  target/test-classes --scan-class-path  --scan-modules  --list-tests --details=verbose


Thanks for using JUnit! Support its development at https://junit.org/sponsoring

├─ JUnit Jupiter
│  ├─ MavenExampleTest
│  │  ├─ shouldRun()
│  │  │       tags: []
│  │  │   uniqueId: [engine:junit-jupiter]/[class:MavenExampleTest]/[method:shouldRun()]
│  │  │     parent: [engine:junit-jupiter]/[class:MavenExampleTest]
│  │  │     source: MethodSource [className = 'MavenExampleTest', methodName = 'shouldRun', methodParameterTypes = '']
│  │  ├─ palindromes(String)
│  │  │       tags: []
│  │  │   uniqueId: [engine:junit-jupiter]/[class:MavenExampleTest]/[test-template:palindromes(java.lang.String)]
│  │  │     parent: [engine:junit-jupiter]/[class:MavenExampleTest]
│  │  │     source: MethodSource [className = 'MavenExampleTest', methodName = 'palindromes', methodParameterTypes = 'java.lang.String']
│  └─ MavenExampleTest
└─ JUnit Jupiter
├─ JUnit Vintage
│       tags: []
│   uniqueId: [engine:junit-vintage]
│     parent: []
├─ JUnit Platform Suite
│       tags: []
│   uniqueId: [engine:junit-platform-suite]
│     parent: []

[         5 containers found ]
[         1 tests found      ]

Test execution:

java -jar junit-platform-console-standalone-1.10.0-20230310.021933-1.jar --class-path  target/test-classes --scan-class-path   --scan-modules  --reports-dir="result" --details=verbose


Thanks for using JUnit! Support its development at https://junit.org/sponsoring

Test plan execution started. Number of static tests: 1
╷
├─ JUnit Jupiter
│  ├─ MavenExampleTest
│  │  ├─ shouldRun()
│  │  │       tags: []
│  │  │   uniqueId: [engine:junit-jupiter]/[class:MavenExampleTest]/[method:shouldRun()]
│  │  │     parent: [engine:junit-jupiter]/[class:MavenExampleTest]
│  │  │     source: MethodSource [className = 'MavenExampleTest', methodName = 'shouldRun', methodParameterTypes = '']
│  │  │   duration: 8 ms
│  │  │     status: ✔ SUCCESSFUL
│  │  ├─ palindromes(String)
│  │  │  ├─ [1] racecar dynamically registered
│  │  │  ├─ [1] racecar
│  │  │  │       tags: []
│  │  │  │   uniqueId: [engine:junit-jupiter]/[class:MavenExampleTest]/[test-template:palindromes(java.lang.String)]/[test-template-invocation:#1]
│  │  │  │     parent: [engine:junit-jupiter]/[class:MavenExampleTest]/[test-template:palindromes(java.lang.String)]
│  │  │  │     source: MethodSource [className = 'MavenExampleTest', methodName = 'palindromes', methodParameterTypes = 'java.lang.String']
│  │  │  │   duration: 8 ms
│  │  │  │     status: ✔ SUCCESSFUL
│  │  │  ├─ [2] radar dynamically registered
│  │  │  ├─ [2] radar
│  │  │  │       tags: []
│  │  │  │   uniqueId: [engine:junit-jupiter]/[class:MavenExampleTest]/[test-template:palindromes(java.lang.String)]/[test-template-invocation:#2]
│  │  │  │     parent: [engine:junit-jupiter]/[class:MavenExampleTest]/[test-template:palindromes(java.lang.String)]
│  │  │  │     source: MethodSource [className = 'MavenExampleTest', methodName = 'palindromes', methodParameterTypes = 'java.lang.String']
│  │  │  │   duration: 1 ms
│  │  │  │     status: ✔ SUCCESSFUL
│  │  │  ├─ [3] able was I ere I saw elba dynamically registered
│  │  │  ├─ [3] able was I ere I saw elba
│  │  │  │       tags: []
│  │  │  │   uniqueId: [engine:junit-jupiter]/[class:MavenExampleTest]/[test-template:palindromes(java.lang.String)]/[test-template-invocation:#3]
│  │  │  │     parent: [engine:junit-jupiter]/[class:MavenExampleTest]/[test-template:palindromes(java.lang.String)]
│  │  │  │     source: MethodSource [className = 'MavenExampleTest', methodName = 'palindromes', methodParameterTypes = 'java.lang.String']
│  │  │  │   duration: 1 ms
│  │  │  │     status: ✔ SUCCESSFUL
│  │  └─ palindromes(String) finished after 23 ms.
│  └─ MavenExampleTest finished after 41 ms.
└─ JUnit Jupiter finished after 128 ms.
├─ JUnit Vintage
└─ JUnit Vintage finished after 7 ms.
├─ JUnit Platform Suite
└─ JUnit Platform Suite finished after 2 ms.
Test plan execution finished. Number of all tests: 4

Test run finished after 159 ms
[         5 containers found      ]
[         0 containers skipped    ]
[         5 containers started    ]
[         0 containers aborted    ]
[         5 containers successful ]
[         0 containers failed     ]
[         4 tests found           ]
[         0 tests skipped         ]
[         4 tests started         ]
[         0 tests aborted         ]
[         4 tests successful      ]
[         0 tests failed          ]

@sbrannen
Copy link
Member

I tried it a bit and found mismatches between the results of test discovery and test execution for the parameterized tests: 1 test after discovery and 4 tests after execution.

That's to be expected.

During the discovery phase only "static" tests can be detected. This means that you won't see dynamic test invocations or parameterized test invocations counted as tests during discovery, but you will see them during the execution phase.

stoyanK7 and others added 20 commits April 14, 2023 12:49
This commit adds a getter for TestPlan's
'allIdentifiers' field.

Issue: #111
Executing 'java -jar junit.jar -cp "./test/" --list-tests'
will return
╷
├─ JUnit Vintage ↷
├─ CarTest ↷
│  ├─ testModel() ↷
│  └─ testMake() ↷
└─ JUnit Jupiter ↷

Issue: #2854
The purpose of this interface is to be able to extend the 3
different details printing listeners which inherit TestExecutionListener:
* TreePrintingListener
* VerboseTreePrintingListener
* FlatPrinting Listener

Issue: #2854
When listing tests, the TreePrinter assumes the test
is skipped and adds the '↷' symbol. If the test has no
duration, result and reason, it is assumed the test
is just being listed and the '↷' is omitted.

Issue: #2854
All previous commits are connected and
the feature is working.

Issue: #2854
Pull request: #2870
@marcphilipp marcphilipp force-pushed the marc/2854-console-launcher-discover branch from 1b628f1 to 753c88f Compare April 14, 2023 10:57
@marcphilipp marcphilipp force-pushed the marc/2854-console-launcher-discover branch from 180b92b to 46930ab Compare April 14, 2023 11:08
@marcphilipp marcphilipp marked this pull request as ready for review April 14, 2023 11:20
@marcphilipp marcphilipp merged commit 30dc835 into main Apr 14, 2023
@marcphilipp marcphilipp deleted the marc/2854-console-launcher-discover branch April 14, 2023 11:22
yhkuo41 pushed a commit to yhkuo41/junit5 that referenced this pull request Apr 19, 2023
The new `--list-tests` option prints the result of test discovery.

Resolves junit-team#2854.

Co-authored-by: stoyanK7 <stoyank127@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add an option to list all tests found by the Console Launcher

5 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.