Skip to content

fix: apply only new or changed databricks_tags on table re-runs#1572

Open
sd-db wants to merge 1 commit into
mainfrom
fix/issue-1308-table-tags-reapplied
Open

fix: apply only new or changed databricks_tags on table re-runs#1572
sd-db wants to merge 1 commit into
mainfrom
fix/issue-1308-table-tags-reapplied

Conversation

@sd-db

@sd-db sd-db commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Problem

For the table materialization, dbt re-applies every configured databricks_tags (table-level) and column-level databricks_tags on every run via ALTER … SET TAGS — one ALTER per column. When many columns are tagged this dominates run time (each SET TAGS is ~300ms) even when nothing changed.

Root cause

The set-only tag diff already exists (TagsConfig.get_diff / ColumnTagsConfig.get_diff) but is wired only into the incremental materialization. The table path applies all configured tags unconditionally on every run — both the v1/legacy path and the v2 create_table_at.

Fix

Reuse the existing diff on the table path. When an existing delta/iceberg table is replaced in place (CREATE OR REPLACE, which preserves tags), fetch the current server tags and apply only those that are new or changed. Fresh creates and drop+recreates apply all tags (no tags to diff against), and tag-free models skip the fetch entirely.

The diff is gated on replaced_in_place — an existing delta/iceberg table replaced in place — not merely on a relation having existed. A drop+recreate (a non-table type, or a non-delta/iceberg format) produces a fresh table with no inherited tags, so it must apply all configured tags rather than diff against possibly-stale metadata.

Tests

New tests/functional/adapter/tags/test_table_tag_fetch_skips.py:

  • a tag-free table never fetches tag metadata;
  • a re-run with table/column tags fetches to diff (v1 and v2 paths);
  • a dropped+recreated relation applies all tags without fetching.

Existing tag and incremental-metadata-fetch suites are unchanged and green.

Closes #1308

@sd-db sd-db requested a review from jprakash-db as a code owner July 1, 2026 12:21
@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  dbt/adapters/databricks
  impl.py 1134, 1141-1150, 1157-1166, 1170
Project Total  

This report was generated by python-coverage-comment-action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Databricks_tags are set every run

2 participants