Tests are failing on main branch #2702
Description
Description of the problem, including code/CLI snippet
Tests are failing on a main branch from a clean install.
Expected Behavior
100% of tests should pass.
Actual Behavior
# cd /tmp
# git clone https://github.com/python-gitlab/python-gitlab.git
Cloning into 'python-gitlab'...
remote: Enumerating objects: 18223, done.
remote: Counting objects: 100% (253/253), done.
remote: Compressing objects: 100% (114/114), done.
remote: Total 18223 (delta 153), reused 208 (delta 138), pack-reused 17970
Receiving objects: 100% (18223/18223), 4.81 MiB | 15.89 MiB/s, done.
Resolving deltas: 100% (13210/13210), done.
# cd python-gitlab
# git rev-parse HEAD
bf68485613756e9916de1bb10c8c4096af4ffd1e
# tox -e py311
py311: install_deps> pip install -r /tmp/python-gitlab/requirements.txt -r /tmp/python-gitlab/requirements-test.txt -e .
py311: commands[0]> pytest tests/unit
=================================================================================================================== test session starts ====================================================================================================================
platform linux -- Python 3.11.0rc1, pytest-7.4.2, pluggy-1.3.0
cachedir: .tox/py311/.pytest_cache
rootdir: /tmp/python-gitlab
configfile: pyproject.toml
plugins: cov-4.1.0, console-scripts-1.4.1, github-actions-annotate-failures-0.2.0
collected 1013 items
tests/unit/test_cli.py ..................... [ 2%]
tests/unit/test_config.py .................. [ 3%]
tests/unit/test_exceptions.py ... [ 4%]
tests/unit/test_gitlab.py ........................................... [ 8%]
tests/unit/test_gitlab_auth.py ............. [ 9%]
tests/unit/test_gitlab_http_methods.py ........................................................ [ 15%]
tests/unit/test_types.py ................ [ 16%]
tests/unit/test_utils.py ....................... [ 19%]
tests/unit/_backends/test_requests_backend.py ......... [ 19%]
tests/unit/base/test_rest_manager.py ... [ 20%]
tests/unit/base/test_rest_object.py ....................................... [ 24%]
tests/unit/meta/test_ensure_type_hints.py .......................................................................................................................................................................................................... [ 44%]
...................................................................................................................................................................................................................................... [ 66%]
tests/unit/meta/test_imports.py .. [ 66%]
tests/unit/meta/test_mro.py .. [ 67%]
tests/unit/mixins/test_meta_mixins.py ... [ 67%]
tests/unit/mixins/test_mixin_methods.py ..................... [ 69%]
tests/unit/mixins/test_object_mixins_attributes.py ...... [ 70%]
tests/unit/objects/test_appearance.py .. [ 70%]
tests/unit/objects/test_applications.py . [ 70%]
tests/unit/objects/test_audit_events.py ...... [ 70%]
tests/unit/objects/test_badges.py ............ [ 72%]
tests/unit/objects/test_bridges.py . [ 72%]
tests/unit/objects/test_bulk_imports.py ...... [ 72%]
tests/unit/objects/test_ci_lint.py ..... [ 73%]
tests/unit/objects/test_commits.py .... [ 73%]
tests/unit/objects/test_deploy_tokens.py . [ 73%]
tests/unit/objects/test_deployments.py ... [ 74%]
tests/unit/objects/test_environments.py .. [ 74%]
tests/unit/objects/test_group_access_tokens.py ... [ 74%]
tests/unit/objects/test_groups.py ........s..s.......... [ 76%]
tests/unit/objects/test_hooks.py .............. [ 78%]
tests/unit/objects/test_invitations.py .......... [ 79%]
tests/unit/objects/test_issues.py ...... [ 79%]
tests/unit/objects/test_iterations.py .. [ 79%]
tests/unit/objects/test_job_artifacts.py .. [ 80%]
tests/unit/objects/test_job_token_scope.py .... [ 80%]
tests/unit/objects/test_jobs.py ... [ 80%]
tests/unit/objects/test_keys.py ... [ 81%]
tests/unit/objects/test_members.py ... [ 81%]
tests/unit/objects/test_merge_request_pipelines.py .. [ 81%]
tests/unit/objects/test_merge_requests.py .. [ 81%]
tests/unit/objects/test_merge_trains.py . [ 81%]
tests/unit/objects/test_packages.py ............... [ 83%]
tests/unit/objects/test_personal_access_tokens.py ....... [ 84%]
tests/unit/objects/test_pipeline_schedules.py ... [ 84%]
tests/unit/objects/test_pipelines.py ..... [ 84%]
tests/unit/objects/test_project_access_tokens.py ... [ 85%]
tests/unit/objects/test_project_import_export.py .......... [ 86%]
tests/unit/objects/test_project_merge_request_approvals.py ......... [ 87%]
tests/unit/objects/test_project_statistics.py . [ 87%]
tests/unit/objects/test_projects.py ............s......................... [ 90%]
tests/unit/objects/test_registry_repositories.py .... [ 91%]
tests/unit/objects/test_releases.py ...... [ 91%]
tests/unit/objects/test_remote_mirrors.py .... [ 92%]
tests/unit/objects/test_repositories.py . [ 92%]
tests/unit/objects/test_resource_groups.py ... [ 92%]
tests/unit/objects/test_resource_iteration_events.py .. [ 92%]
tests/unit/objects/test_resource_label_events.py ... [ 93%]
tests/unit/objects/test_resource_milestone_events.py .. [ 93%]
tests/unit/objects/test_resource_state_events.py .... [ 93%]
tests/unit/objects/test_runners.py ............. [ 95%]
tests/unit/objects/test_secure_files.py .... [ 95%]
tests/unit/objects/test_services.py ..... [ 95%]
tests/unit/objects/test_snippets.py ... [ 96%]
tests/unit/objects/test_statistics.py . [ 96%]
tests/unit/objects/test_submodules.py . [ 96%]
tests/unit/objects/test_todos.py .. [ 96%]
tests/unit/objects/test_topics.py ...... [ 97%]
tests/unit/objects/test_users.py ............. [ 98%]
tests/unit/objects/test_variables.py ............... [100%]
============================================================================================================= 1010 passed, 3 skipped in 4.47s ==============================================================================================================
py311: OK (14.39=setup[9.36]+cmd[5.03] seconds)
congratulations :) (14.42 seconds)
# tox -e api_func_v4
api_func_v4: install_deps> pip install -r /tmp/python-gitlab/requirements-docker.txt -e .
api_func_v4: commands[0]> pytest --cov --cov-report xml tests/functional/api
=================================================================================================================== test session starts ====================================================================================================================
platform linux -- Python 3.11.0rc1, pytest-7.4.2, pluggy-1.3.0
cachedir: .tox/api_func_v4/.pytest_cache
rootdir: /tmp/python-gitlab
configfile: pyproject.toml
plugins: docker-2.0.1, cov-4.1.0, console-scripts-1.4.1, github-actions-annotate-failures-0.2.0
collected 192 items
Starting GitLab container.
Waiting for GitLab to reconfigure.
This will take a few minutes.
tests/functional/api/test_boards.py .. [ 1%]
tests/functional/api/test_branches.py . [ 1%]
tests/functional/api/test_bulk_imports.py . [ 2%]
tests/functional/api/test_current_user.py .... [ 4%]
tests/functional/api/test_deploy_keys.py . [ 4%]
tests/functional/api/test_deploy_tokens.py .. [ 5%]
tests/functional/api/test_epics.py .x. [ 7%]
tests/functional/api/test_gitlab.py .............................s...... [ 26%]
tests/functional/api/test_groups.py ..x.........x [ 32%]
tests/functional/api/test_import_export.py .... [ 34%]
tests/functional/api/test_issues.py ........ [ 39%]
tests/functional/api/test_keys.py .. [ 40%]
tests/functional/api/test_lazy_objects.py ..... [ 42%]
tests/functional/api/test_merge_requests.py .........x........ [ 52%]
tests/functional/api/test_packages.py .......... [ 57%]
tests/functional/api/test_projects.py .....s...................... [ 71%]
tests/functional/api/test_push_rules.py . [ 72%]
tests/functional/api/test_releases.py ..... [ 75%]
tests/functional/api/test_repository.py ................... [ 84%]
tests/functional/api/test_services.py ..... [ 87%]
tests/functional/api/test_snippets.py .... [ 89%]
tests/functional/api/test_statistics.py . [ 90%]
tests/functional/api/test_topics.py . [ 90%]
tests/functional/api/test_users.py .............. [ 97%]
tests/functional/api/test_variables.py ... [ 99%]
tests/functional/api/test_wikis.py . [100%]
-------- coverage: platform linux, python 3.11.0-candidate-1 ---------
Coverage XML written to file coverage.xml
================================================================================================== 186 passed, 2 skipped, 4 xfailed in 707.64s (0:11:47) ===================================================================================================
api_func_v4: OK (716.05=setup[7.96]+cmd[708.09] seconds)
congratulations :) (716.09 seconds)
# tox -e cli_func_v4
cli_func_v4: install_deps> pip install -r /tmp/python-gitlab/requirements-docker.txt -e .
cli_func_v4: commands[0]> pytest --script-launch-mode=subprocess --cov --cov-report xml tests/functional/cli
=================================================================================================================== test session starts ====================================================================================================================
platform linux -- Python 3.11.0rc1, pytest-7.4.2, pluggy-1.3.0
cachedir: .tox/cli_func_v4/.pytest_cache
rootdir: /tmp/python-gitlab
configfile: pyproject.toml
plugins: docker-2.0.1, cov-4.1.0, console-scripts-1.4.1, github-actions-annotate-failures-0.2.0
collected 99 items
Starting GitLab container.
Waiting for GitLab to reconfigure.
This will take a few minutes.
tests/functional/cli/test_cli.py ...FFF................ [ 22%]
tests/functional/cli/test_cli_artifacts.py ... [ 25%]
tests/functional/cli/test_cli_packages.py .... [ 29%]
tests/functional/cli/test_cli_projects.py ... [ 32%]
tests/functional/cli/test_cli_repository.py ...... [ 38%]
tests/functional/cli/test_cli_resource_access_tokens.py ..s. [ 42%]
tests/functional/cli/test_cli_users.py .. [ 44%]
tests/functional/cli/test_cli_v4.py .................................................. [ 94%]
tests/functional/cli/test_cli_variables.py ..... [100%]
========================================================================================================================= FAILURES =========================================================================================================================
_________________________________________________________________________________________________ test_global_help_prints_resources_vertically[subprocess] _________________________________________________________________________________________________
script_runner = <ScriptRunner subprocess>
def test_global_help_prints_resources_vertically(script_runner):
ret = script_runner.run(["gitlab", "--help"])
> assert """resource:\n application\n application-appearance\n""" in ret.stdout
E AssertionError: assert 'resource:\n application\n application-appearance\n' in 'usage: gitlab [--version] [-v] [-d] [-c CONFIG_FILE] [-g GITLAB] [-o {json,legacy,yaml}] [-f FIELDS] [--server-url SE...r: GITLAB_OAUTH_TOKEN]\n --job-token JOB_TOKEN\n GitLab CI job token [env var: CI_JOB_TOKEN]\n'
E + where 'usage: gitlab [--version] [-v] [-d] [-c CONFIG_FILE] [-g GITLAB] [-o {json,legacy,yaml}] [-f FIELDS] [--server-url SE...r: GITLAB_OAUTH_TOKEN]\n --job-token JOB_TOKEN\n GitLab CI job token [env var: CI_JOB_TOKEN]\n' = <pytest_console_scripts.RunResult object at 0x7f5cd6961d50>.stdout
tests/functional/cli/test_cli.py:39: AssertionError
------------------------------------------------------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------------------------------------------------------
# Running console script: ['gitlab', '--help']
# Script return code: 0
# Script stdout:
usage: gitlab [--version] [-v] [-d] [-c CONFIG_FILE] [-g GITLAB] [-o {json,legacy,yaml}] [-f FIELDS] [--server-url SERVER_URL] [--ssl-verify SSL_VERIFY] [--timeout TIMEOUT] [--api-version API_VERSION] [--per-page PER_PAGE] [--pagination PAGINATION]
[--order-by ORDER_BY] [--user-agent USER_AGENT] [--private-token PRIVATE_TOKEN | --oauth-token OAUTH_TOKEN | --job-token JOB_TOKEN]
GitLab API Command Line Interface
options:
--version Display the version.
-v, --verbose, --fancy
Verbose mode (legacy format only) [env var: GITLAB_VERBOSE]
-d, --debug Debug mode (display HTTP requests) [env var: GITLAB_DEBUG]
-c CONFIG_FILE, --config-file CONFIG_FILE
Configuration file to use. Can be used multiple times. [env var: PYTHON_GITLAB_CFG]
-g GITLAB, --gitlab GITLAB
Which configuration section should be used. If not defined, the default selection will be used.
-o {json,legacy,yaml}, --output {json,legacy,yaml}
Output format (v4 only): json|legacy|yaml
-f FIELDS, --fields FIELDS
Fields to display in the output (comma separated). Not used with legacy output
--server-url SERVER_URL
GitLab server URL [env var: GITLAB_URL]
--ssl-verify SSL_VERIFY
Whether SSL certificates should be validated. [env var: GITLAB_SSL_VERIFY]
--timeout TIMEOUT Timeout to use for requests to the GitLab server. [env var: GITLAB_TIMEOUT]
--api-version API_VERSION
GitLab API version [env var: GITLAB_API_VERSION]
--per-page PER_PAGE Number of entries to return per page in the response. [env var: GITLAB_PER_PAGE]
--pagination PAGINATION
Whether to use keyset or offset pagination [env var: GITLAB_PAGINATION]
--order-by ORDER_BY Set order_by globally [env var: GITLAB_ORDER_BY]
--user-agent USER_AGENT
The user agent to send to GitLab with the HTTP request. [env var: GITLAB_USER_AGENT]
--private-token PRIVATE_TOKEN
GitLab private access token [env var: GITLAB_PRIVATE_TOKEN]
--oauth-token OAUTH_TOKEN
GitLab OAuth token [env var: GITLAB_OAUTH_TOKEN]
--job-token JOB_TOKEN
GitLab CI job token [env var: CI_JOB_TOKEN]
# Script stderr:
_________________________________________________________________________________________________ test_resource_help_prints_actions_vertically[subprocess] _________________________________________________________________________________________________
script_runner = <ScriptRunner subprocess>
def test_resource_help_prints_actions_vertically(script_runner):
ret = script_runner.run(["gitlab", "project", "--help"])
> assert """action:\n list\n get""" in ret.stdout
E AssertionError: assert 'action:\n list\n get' in 'usage: gitlab [--version] [-v] [-d] [-c CONFIG_FILE] [-g GITLAB] [-o {json,legacy,yaml}] [-f FIELDS] [--server-url SE...r: GITLAB_OAUTH_TOKEN]\n --job-token JOB_TOKEN\n
GitLab CI job token [env var: CI_JOB_TOKEN]\n'
E + where 'usage: gitlab [--version] [-v] [-d] [-c CONFIG_FILE] [-g GITLAB] [-o {json,legacy,yaml}] [-f FIELDS] [--server-url SE...r: GITLAB_OAUTH_TOKEN]\n --job-token JOB_TOKEN\n GitLab CI job token [env var: CI_JOB_TOKEN]\n' = <pytest_console_scripts.RunResult object at 0x7f5cd679ef50>.stdout
tests/functional/cli/test_cli.py:45: AssertionError
------------------------------------------------------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------------------------------------------------------
# Running console script: ['gitlab', 'project', '--help']
# Script return code: 0
# Script stdout:
usage: gitlab [--version] [-v] [-d] [-c CONFIG_FILE] [-g GITLAB] [-o {json,legacy,yaml}] [-f FIELDS] [--server-url SERVER_URL] [--ssl-verify SSL_VERIFY] [--timeout TIMEOUT] [--api-version API_VERSION] [--per-page PER_PAGE] [--pagination PAGINATION]
[--order-by ORDER_BY] [--user-agent USER_AGENT] [--private-token PRIVATE_TOKEN | --oauth-token OAUTH_TOKEN | --job-token JOB_TOKEN]
GitLab API Command Line Interface
options:
--version Display the version.
-v, --verbose, --fancy
Verbose mode (legacy format only) [env var: GITLAB_VERBOSE]
-d, --debug Debug mode (display HTTP requests) [env var: GITLAB_DEBUG]
-c CONFIG_FILE, --config-file CONFIG_FILE
Configuration file to use. Can be used multiple times. [env var: PYTHON_GITLAB_CFG]
-g GITLAB, --gitlab GITLAB
Which configuration section should be used. If not defined, the default selection will be used.
-o {json,legacy,yaml}, --output {json,legacy,yaml}
Output format (v4 only): json|legacy|yaml
-f FIELDS, --fields FIELDS
Fields to display in the output (comma separated). Not used with legacy output
--server-url SERVER_URL
GitLab server URL [env var: GITLAB_URL]
--ssl-verify SSL_VERIFY
Whether SSL certificates should be validated. [env var: GITLAB_SSL_VERIFY]
--timeout TIMEOUT Timeout to use for requests to the GitLab server. [env var: GITLAB_TIMEOUT]
--api-version API_VERSION
GitLab API version [env var: GITLAB_API_VERSION]
--per-page PER_PAGE Number of entries to return per page in the response. [env var: GITLAB_PER_PAGE]
--pagination PAGINATION
Whether to use keyset or offset pagination [env var: GITLAB_PAGINATION]
--order-by ORDER_BY Set order_by globally [env var: GITLAB_ORDER_BY]
--user-agent USER_AGENT
The user agent to send to GitLab with the HTTP request. [env var: GITLAB_USER_AGENT]
--private-token PRIVATE_TOKEN
GitLab private access token [env var: GITLAB_PRIVATE_TOKEN]
--oauth-token OAUTH_TOKEN
GitLab OAuth token [env var: GITLAB_OAUTH_TOKEN]
--job-token JOB_TOKEN
GitLab CI job token [env var: CI_JOB_TOKEN]
# Script stderr:
_________________________________________________________________________________________ test_resource_help_prints_actions_vertically_only_one_action[subprocess] _________________________________________________________________________________________
script_runner = <ScriptRunner subprocess>
def test_resource_help_prints_actions_vertically_only_one_action(script_runner):
ret = script_runner.run(["gitlab", "event", "--help"])
> assert """action:\n list\n""" in ret.stdout
E AssertionError: assert 'action:\n list\n' in 'usage: gitlab [--version] [-v] [-d] [-c CONFIG_FILE] [-g GITLAB] [-o {json,legacy,yaml}] [-f FIELDS] [--server-url SE...r: GITLAB_OAUTH_TOKEN]\n --job-token JOB_TOKEN\n GitLab CI job token [env var: CI_JOB_TOKEN]\n'
E + where 'usage: gitlab [--version] [-v] [-d] [-c CONFIG_FILE] [-g GITLAB] [-o {json,legacy,yaml}] [-f FIELDS] [--server-url SE...r: GITLAB_OAUTH_TOKEN]\n --job-token JOB_TOKEN\n GitLab CI job token [env var: CI_JOB_TOKEN]\n' = <pytest_console_scripts.RunResult object at 0x7f5cd693a6d0>.stdout
tests/functional/cli/test_cli.py:51: AssertionError
------------------------------------------------------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------------------------------------------------------
# Running console script: ['gitlab', 'event', '--help']
# Script return code: 0
# Script stdout:
usage: gitlab [--version] [-v] [-d] [-c CONFIG_FILE] [-g GITLAB] [-o {json,legacy,yaml}] [-f FIELDS] [--server-url SERVER_URL] [--ssl-verify SSL_VERIFY] [--timeout TIMEOUT] [--api-version API_VERSION] [--per-page PER_PAGE] [--pagination PAGINATION]
[--order-by ORDER_BY] [--user-agent USER_AGENT] [--private-token PRIVATE_TOKEN | --oauth-token OAUTH_TOKEN | --job-token JOB_TOKEN]
GitLab API Command Line Interface
options:
--version Display the version.
-v, --verbose, --fancy
Verbose mode (legacy format only) [env var: GITLAB_VERBOSE]
-d, --debug Debug mode (display HTTP requests) [env var: GITLAB_DEBUG]
-c CONFIG_FILE, --config-file CONFIG_FILE
Configuration file to use. Can be used multiple times. [env var: PYTHON_GITLAB_CFG]
-g GITLAB, --gitlab GITLAB
Which configuration section should be used. If not defined, the default selection will be used.
-o {json,legacy,yaml}, --output {json,legacy,yaml}
Output format (v4 only): json|legacy|yaml
-f FIELDS, --fields FIELDS
Fields to display in the output (comma separated). Not used with legacy output
--server-url SERVER_URL
GitLab server URL [env var: GITLAB_URL]
--ssl-verify SSL_VERIFY
Whether SSL certificates should be validated. [env var: GITLAB_SSL_VERIFY]
--timeout TIMEOUT Timeout to use for requests to the GitLab server. [env var: GITLAB_TIMEOUT]
--api-version API_VERSION
GitLab API version [env var: GITLAB_API_VERSION]
--per-page PER_PAGE Number of entries to return per page in the response. [env var: GITLAB_PER_PAGE]
--pagination PAGINATION
Whether to use keyset or offset pagination [env var: GITLAB_PAGINATION]
--order-by ORDER_BY Set order_by globally [env var: GITLAB_ORDER_BY]
--user-agent USER_AGENT
The user agent to send to GitLab with the HTTP request. [env var: GITLAB_USER_AGENT]
--private-token PRIVATE_TOKEN
GitLab private access token [env var: GITLAB_PRIVATE_TOKEN]
--oauth-token OAUTH_TOKEN
GitLab OAuth token [env var: GITLAB_OAUTH_TOKEN]
--job-token JOB_TOKEN
GitLab CI job token [env var: CI_JOB_TOKEN]
# Script stderr:
-------- coverage: platform linux, python 3.11.0-candidate-1 ---------
Coverage XML written to file coverage.xml
================================================================================================================= short test summary info ==================================================================================================================
FAILED tests/functional/cli/test_cli.py::test_global_help_prints_resources_vertically[subprocess] - AssertionError: assert 'resource:\n application\n application-appearance\n' in 'usage: gitlab [--version] [-v] [-d] [-c CONFIG_FILE] [-g GITLAB] [-o {json,legacy,yaml}] [-f FIELDS] [--server-url SE...r: GITLAB_OAUTH_TOKEN]\n --job-token JOB_TOK...
FAILED tests/functional/cli/test_cli.py::test_resource_help_prints_actions_vertically[subprocess] - AssertionError: assert 'action:\n list\n get' in 'usage: gitlab [--version] [-v] [-d] [-c CONFIG_FILE] [-g GITLAB] [-o {json,legacy,yaml}] [-f FIELDS] [--server-url SE...r: GITLAB_OAUTH_TOKEN]\n --job-token JOB_TOKEN\n Gi...
FAILED tests/functional/cli/test_cli.py::test_resource_help_prints_actions_vertically_only_one_action[subprocess] - AssertionError: assert 'action:\n list\n' in 'usage: gitlab [--version] [-v] [-d] [-c CONFIG_FILE] [-g GITLAB] [-o {json,legacy,yaml}] [-f FIELDS] [--server-url SE...r: GITLAB_OAUTH_TOKEN]\n --job-token JOB_TOKEN\n GitLab ...
=================================================================================================== 3 failed, 95 passed, 1 skipped in 507.92s (0:08:27) ====================================================================================================
cli_func_v4: exit 1 (508.87 seconds) /tmp/python-gitlab> pytest --script-launch-mode=subprocess --cov --cov-report xml tests/functional/cli pid=94781
cli_func_v4: FAIL code 1 (518.18=setup[9.31]+cmd[508.87] seconds)
evaluation failed :( (518.22 seconds)
#
Got 3 failures.
Specifications
- python-gitlab version: Current
mainbranch: bf68485 - API version you are using (v3/v4): N/A
- Gitlab server version (or gitlab.com): 15.4.0-ee.0 (specified in tests/functional/fixtures/.env
GITLAB_IMAGE=gitlab/gitlab-ee
GITLAB_TAG=15.4.0-ee.0