Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Document unpartitioned Hive insert_overwrite behavior for Athena #9643

Copy link
Copy link

Description

@AlejandroMorgante
Issue body actions

Contributions

  • I have read the contribution docs, and understand what's expected of me.

Link to the page on docs.getdbt.com requiring updates

https://docs.getdbt.com/reference/resource-configs/athena-configs

What part(s) of the page would you like to see updated?

The "Incremental table models" section currently documents insert_overwrite as always falling back to append when no partitions are defined.

dbt-labs/dbt-adapters#2082 proposes distinguishing an omitted strategy from an explicitly configured insert_overwrite for unpartitioned Hive models:

  • When the strategy is omitted, preserve the existing append fallback.
  • When insert_overwrite is explicit and dbt creates a new S3 location per run, which is the default, replace the complete table.
  • When the model reuses an S3 path through external_location or s3_data_naming, preserve append and emit an informational message.
  • Preserve existing behavior for partitioned Hive models and all other strategies.

The Athena configuration page should document this distinction when the adapter change is released.

Suggested replacement copy

Replace the current insert_overwrite bullet with:

  • insert_overwrite (default for Hive): For partitioned models, dbt deletes overlapping partitions from the destination table and inserts the new records. For unpartitioned models, an explicitly configured incremental_strategy='insert_overwrite' replaces the complete table when dbt creates a new S3 location per run (s3_data_naming contains unique, which is the default, and external_location is not configured). If the strategy is omitted, dbt preserves the historical append fallback. Models that explicitly request insert_overwrite but reuse the same S3 path also preserve append and emit an informational message.

Additional information

Feature issue: dbt-labs/dbt-adapters#2081

Implementation PR: dbt-labs/dbt-adapters#2082

The dbt-athena README is also updated in the implementation PR.

Reactions are currently unavailable

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    Morty Proxy This is a proxified and sanitized view of the page, visit original site.