Skip to content

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

TestNg Setup/Before errors should NOT be separate test cases in report #643

Copy link
Copy link
Open
@bgalamb

Description

@bgalamb
Issue body actions

Describe the bug
Allure generates individual test case from testNG setup/before methods too if they break. Attached project to reproduce:
bug_reports.zip

To Reproduce
Steps to reproduce the behavior:

  1. unzip Attached project
  2. execute mvn clean test
  3. execute cd target and allure serve
  4. Check generated report. (I attached screenshot too)
  5. It contains one additional test "myBeforeClass" which is broken. This confuses testers because 1-the number of tests may be varying in the test class 2-it links a method which may be totally unknown for them (if its located in parent or static class).
  6. 3-Also It does not add too much value because the same stack can be found linked to the "real" test.

Expected behavior
I would NOT expect to see the setup/before hook's excpetion as independent tests in the report.
I suggest deleting the onConfigurationFailure() method from io/qameta/allure/testng/AllureTestNg.java.
Follow steps to verify:
11. Open attached project
12. comment or delete method onConfigurationFailure() in io/qameta/allure/testng/AllureTestNg.java
13. repeat steps 2,3
14. See that in this case only the real broken test is shown in the list. And we can also find the error starting from this.

BTW: the comment in code also indicates that configuration error should not be separate tests

Screenshots
If applicable, add screenshots to help explain your problem.
Screenshot 2021-08-30 at 14 35 30
Screenshot 2021-08-30 at 14 36 56
5b. if uploaded to allureEE it generated strange orphan tests.

Screenshot 2021-08-30 at 14 46 35

==expected images===

Screenshot 2021-08-30 at 14 33 35

Screenshot 2021-08-30 at 14 33 48

Desktop (please complete the following information):

  • OS: [MAC]
  • Browser [chrome]
  • Version [e.g. 22]

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Morty Proxy This is a proxified and sanitized view of the page, visit original site.