Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

[airflow] Update AIR301 and AIR311 with the latest Airflow implementations #17985

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
May 19, 2025

Conversation

Lee-W
Copy link
Contributor

@Lee-W Lee-W commented May 9, 2025

Summary

  • Remove the following rules
    • name
      • airflow.auth.managers.base_auth_manager.is_authorized_datasetairflow.api_fastapi.auth.managers.base_auth_manager.is_authorized_asset
      • airflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_datasetairflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_asset
  • Update the following rules
    • name
      • airflow.models.baseoperatorlink.BaseOperatorLinkairflow.sdk.BaseOperatorLink
      • airflow.api_connexion.security.requires_access → "Use airflow.api_fastapi.core_api.security.requires_access_* instead`"
      • airflow.api_connexion.security.requires_access_datasetairflow.api_fastapi.core_api.security.requires_access_asset
      • airflow.notifications.basenotifier.BaseNotifierairflow.sdk.bases.notifier.BaseNotifier
      • airflow.www.auth.has_access → None
      • airflow.www.auth.has_access_dataset → None
      • airflow.www.utils.get_sensitive_variables_fields→ None
      • airflow.www.utils.should_hide_value_for_key→ None
    • class attribute
      • airflow..sensors.weekday.DayOfWeekSensor
        • use_task_execution_day removed
      • airflow.providers.amazon.aws.auth_manager.aws_auth_manager.AwsAuthManager
        • is_authorized_dataset
  • Add the following rules
    • class attribute
      • airflow.auth.managers.base_auth_manager.BaseAuthManager | airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager
    • name
      • airflow.auth.managers.base_auth_manager.BaseAuthManagerairflow.api_fastapi.auth.managers.base_auth_manager.BaseAuthManager * is_authorized_datasetis_authorized_asset
  • refactor
    • simplify unnecessary match with if else
    • rename Replacement::Name as Replacement::AttrName

Test Plan

The test fixtures have been revised and updated.

@Lee-W Lee-W mentioned this pull request May 9, 2025
2 tasks
@Lee-W Lee-W changed the title Update air301 air311 rules [airflow] Update AIR301, AIR311 rules with the latest Airflow implemnetation May 9, 2025
@Lee-W Lee-W force-pushed the update-AIR301-AIR311-rules branch 2 times, most recently from dedaab9 to 4e3f5ef Compare May 12, 2025 15:00
Copy link
Contributor

github-actions bot commented May 12, 2025

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+21 -2 violations, +0 -0 fixes in 1 projects; 54 projects unchanged)

apache/airflow (+21 -2 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --no-fix --output-format concise --preview --select ALL

+ dev/airflow_perf/scheduler_dag_execution_timing.py:255:10: AIR301 `airflow.utils.db.create_session` is removed in Airflow 3.0
- dev/airflow_perf/scheduler_dag_execution_timing.py:255:13: AIR301 `airflow.utils.db.create_session` is removed in Airflow 3.0
+ dev/airflow_perf/scheduler_dag_execution_timing.py:302:18: AIR301 `airflow.utils.db.create_session` is removed in Airflow 3.0
- dev/airflow_perf/scheduler_dag_execution_timing.py:302:21: AIR301 `airflow.utils.db.create_session` is removed in Airflow 3.0
+ providers/amazon/src/airflow/providers/amazon/aws/links/base_aws.py:41:19: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/databricks/src/airflow/providers/databricks/operators/databricks.py:250:28: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/databricks/src/airflow/providers/databricks/plugins/databricks_workflow.py:238:26: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/databricks/src/airflow/providers/databricks/plugins/databricks_workflow.py:272:38: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/databricks/src/airflow/providers/databricks/plugins/databricks_workflow.py:366:39: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/dbt/cloud/src/airflow/providers/dbt/cloud/operators/dbt.py:50:34: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/google/src/airflow/providers/google/cloud/links/base.py:38:22: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/google/src/airflow/providers/google/cloud/links/datafusion.py:44:22: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/google/src/airflow/providers/google/cloud/links/dataproc.py:136:24: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/google/src/airflow/providers/google/cloud/links/dataproc.py:83:20: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/google/src/airflow/providers/google/cloud/operators/dataproc_metastore.py:112:37: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/google/src/airflow/providers/google/cloud/operators/dataproc_metastore.py:64:29: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/google/src/airflow/providers/google/marketing_platform/links/analytics_admin.py:39:31: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/microsoft/azure/src/airflow/providers/microsoft/azure/operators/data_factory.py:52:53: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/microsoft/azure/src/airflow/providers/microsoft/azure/operators/powerbi.py:46:19: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/microsoft/azure/src/airflow/providers/microsoft/azure/operators/synapse.py:133:35: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/standard/src/airflow/providers/standard/operators/trigger_dagrun.py:81:25: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/standard/src/airflow/providers/standard/sensors/external_task.py:72:23: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.
+ providers/yandex/src/airflow/providers/yandex/links/yq.py:43:14: AIR311 `airflow.models.baseoperatorlink.BaseOperatorLink` is removed in Airflow 3.0; It still works in Airflow 3.0 but is expected to be removed in a future version.

Changes by rule (2 rules affected)

code total + violation - violation + fix - fix
AIR311 19 19 0 0 0
AIR301 4 2 2 0 0

@ntBre ntBre self-requested a review May 13, 2025 01:40
@Lee-W Lee-W force-pushed the update-AIR301-AIR311-rules branch from 7c2b1e3 to bbaae8e Compare May 15, 2025 03:13
@Lee-W Lee-W marked this pull request as ready for review May 15, 2025 08:44
@Lee-W Lee-W force-pushed the update-AIR301-AIR311-rules branch from bbaae8e to a455884 Compare May 15, 2025 10:35
@ntBre ntBre added rule Implementing or modifying a lint rule preview Related to preview mode features labels May 15, 2025
Copy link
Contributor

@ntBre ntBre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This overall looks good to me. I had a few minor suggestions and one question about the import code.

crates/ruff_linter/src/rules/airflow/rules/removal_in_3.rs Outdated Show resolved Hide resolved
crates/ruff_linter/src/rules/airflow/rules/removal_in_3.rs Outdated Show resolved Hide resolved
crates/ruff_linter/src/rules/airflow/rules/removal_in_3.rs Outdated Show resolved Hide resolved
crates/ruff_linter/src/rules/airflow/rules/removal_in_3.rs Outdated Show resolved Hide resolved
diagnostic.try_set_fix(|| {
let (import_edit, binding) = checker.importer().get_or_import_symbol(
&ImportRequest::import_from(module, name),
let (import_edit, _) = checker.importer().get_or_import_symbol(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're supposed to use the binding returned by get_or_import_symbol. Was something going wrong here when you did that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There 2 cases like the following that need to be handled this way.

from airflow.providers.amazon.aws.auth_manager.avp.entities import AvpEntities

AvpEntities.DATASET

which should be fixed as

from airflow.providers.amazon.aws.auth_manager.avp.entities import AvpEntities

AvpEntities.ASSET

is we use binding, it will be fixed as

from airflow.providers.amazon.aws.auth_manager.avp.entities import AvpEntities

AvpEntities

which is wrong

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I think that makes sense. Thanks!

@Lee-W Lee-W force-pushed the update-AIR301-AIR311-rules branch 3 times, most recently from f61f06c to d75296e Compare May 16, 2025 13:42
Lee-W added 8 commits May 19, 2025 23:08
* Remove the following rules
    * name
        * `airflow.auth.managers.base_auth_manager.is_authorized_dataset` → `airflow.api_fastapi.auth.managers.base_auth_manager.is_authorized_asset`
        * `airflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_dataset` → `airflow.providers.fab.auth_manager.fab_auth_manager.is_authorized_asset`
* Update the following rules
    * name
        * `airflow.api_connexion.security.requires_access` → "Use `airflow.api_fastapi.core_api.security.requires_access_*` instead`"
        * `airflow.api_connexion.security.requires_access_dataset`→ `airflow.api_fastapi.core_api.security.requires_access_asset`
        * `airflow.notifications.basenotifier.BaseNotifier` → `airflow.sdk.bases.notifier.BaseNotifier`
        * `airflow.www.auth.has_access`  → None
        * `airflow.www.auth.has_access_dataset` → None
        * `airflow.www.utils.get_sensitive_variables_fields`→ None
        * `airflow.www.utils.should_hide_value_for_key`→ None
* Add the following rules
    * class attribute
        * `airflow.auth.managers.base_auth_manager.BaseAuthManager` | `airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager`
            * `is_authorized_dataset` → `is_authorized_asset`
* Update the following rules
    * class attribute
        * `airflow..sensors.weekday.DayOfWeekSensor`
            * `use_task_execution_day` removed
        * `airflow.providers.amazon.aws.auth_manager.aws_auth_manager.AwsAuthManager`
            * `is_authorized_dataset`
* Add the following rules
    * name
        * `airflow.auth.managers.base_auth_manager.BaseAuthManager` → `airflow.api_fastapi.auth.managers.base_auth_manager.BaseAuthManager`
@Lee-W Lee-W force-pushed the update-AIR301-AIR311-rules branch from d75296e to d631292 Compare May 19, 2025 15:08
@ntBre ntBre changed the title [airflow] Update AIR301, AIR311 rules with the latest Airflow implemnetation [airflow] Update AIR301 and AIR311 with the latest Airflow implementations May 19, 2025
@ntBre ntBre merged commit 236633c into astral-sh:main May 19, 2025
34 checks passed
dcreager added a commit that referenced this pull request May 19, 2025
…rals

* origin/main:
  [ty] Add hint that PEP 604 union syntax is only available in 3.10+ (#18192)
  Unify `Message` variants (#18051)
  [`airflow`] Update `AIR301` and `AIR311` with the latest Airflow implementations (#17985)
  [`airflow`] Move rules from `AIR312` to `AIR302` (#17940)
  [ty] Small LSP cleanups (#18201)
  [ty] Show related information in diagnostic (#17359)
  Default `src.root` to `['.', '<project_name>']` if the directory exists (#18141)
dcreager added a commit that referenced this pull request May 19, 2025
* main:
  [ty] Use first matching constructor overload when inferring specializations (#18204)
  [ty] Add hint that PEP 604 union syntax is only available in 3.10+ (#18192)
  Unify `Message` variants (#18051)
  [`airflow`] Update `AIR301` and `AIR311` with the latest Airflow implementations (#17985)
  [`airflow`] Move rules from `AIR312` to `AIR302` (#17940)
  [ty] Small LSP cleanups (#18201)
  [ty] Show related information in diagnostic (#17359)
  Default `src.root` to `['.', '<project_name>']` if the directory exists (#18141)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
preview Related to preview mode features rule Implementing or modifying a lint rule
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.