Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Report for skip is a child for unexecuted fixtures. #791

Copy link
Copy link
Open
@dmakhno

Description

@dmakhno
Issue body actions

I'm submitting a ...

  • bug report
  • feature request
  • support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

Skip Tests are reported like they had actions/steps.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

Generate report for with -s:

import pytest
import allure

@pytest.fixture(scope='module', autouse=True)
def module_dep(request: pytest.FixtureRequest):
    with allure.step(f'module start {request.node.nodeid}'):
        pass
    print('<<<  start  >>>')
    yield
    with allure.step(f'module end {request.node.nodeid}'):
        pass
    print('<<</  end  >>>')

@pytest.fixture(autouse=True)
def tc_dep(request: pytest.FixtureRequest):
    with allure.step(f'test start {request.node.nodeid}'):
        pass
    print('<< start test >>')
    yield
    with allure.step(f'test end {request.node.nodeid}'):
        pass
    print('<</ end test >>')


class TestFoo:

    @pytest.mark.skip
    def test_a(self):
        pass

    def test_b(self):
        pass

Observation:

  • test_a has fixtures steps that does not own
  • pytest terminal reports at the time of skip - nothing was run

image

I assume the bug is in allure python due to

{
    "uuid": "1adbda40-73cc-47ab-a3f5-154adeab99db",
    "children": [
        "d0c0a1b8-6b3e-496d-8758-e599ed9facf0",
        "a16e7ffe-92f7-4212-ab95-6dc6486661ab"
    ],
    "befores": [
        {
            "name": "tc_dep",
            "status": "passed",
            "steps": [
                {
                    "name": "test start test_session_skip.py::TestFoo::test_b",
                    "status": "passed"
                }
            ]
        }
    ],
    "afters": [
        {
            "name": "tc_dep::0",
            "status": "passed",
            "steps": [
                {
                    "name": "test end test_session_skip.py::TestFoo::test_b",
                    "status": "passed"
                }
            ]
        }
    ]
}

where children are test_a and test_b - I'd expect only test_b should be among children. Sorry hardly understand allure models. (ps: start/stop removed for a focused view.)

Long thinking on it, I asume there might be reason why each test shows e.g. "module" or "session" setup/teardown fixture. Would be nice if it can be visually clear "shared" parents.

What is the expected behavior?

Nodes/containers do not have nodes that do not belong.

What is the motivation / use case for changing the behavior?

Bug. This is very confusing. And making think that unconditionally skipped test somehow executed fixtures

I spent couple hours in prod environment checking "why do I see it", while provided steps above - explained anomaly.

For a heavy setup/tear down, visual feedback for "children" a bit confusing.

A bit offtopic...

pytest execution protocol:
- module fixture - setup
  - class fixture - setup
    - test fixture - setup
      - test_foo
    - test fixture - teardown
    - test fixture - setup
      - test_baz
    - test fixture - teardown
  - class fixture - teardown
- module fixture - teardown

in allure report shared "class", "module", "package", "session" are shown with each test... and it seems from this issue "function" sometimes too. Nice to have visual indication of shared nodes. Glamour-allure tries to do it (here) but I cannot aford more dependencies and nice if it can be considered "from the box". I'd really like to try to understand is this Item was the first and started "higher-scope" fixtures, or it is just "reported here" (a pointer :) ).

Please tell us about your environment:

  • Allure version: 2.26 or 2.27 (tried both from https://github.com/TobiX/allure-cli via sh script docker run --rm -v $PWD:/work -it tobix/allure-cli $@)
  • Test framework: pytest@7.4.4 or pytest@8.0
  • Allure adaptor: allure-pytest@2.13.2

Other information

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Morty Proxy This is a proxified and sanitized view of the page, visit original site.