Deprecating and removing old dbt model versions

1.Setting a deprecation_date

A final model that other teams, dashboards and external systems query behaves like an API. Its owner must change it eventually, but the people downstream need their queries to keep working. Deprecation is how a producer retires the old shape of such a model on an announced timeline instead of breaking those consumers without warning.

That retirement usually happens through model versions. A version is a separate implementation of one model, kept in the same repository and deployed into the same environment as the others. By convention the file and the warehouse relation are named <model_name>_v<v>, so dim_customers_v1 and dim_customers_v2 can both exist while consumers keep calling ref('dim_customers').

deprecation_date is a model property that tells consumers when a model, or one version of it, will stop being supported. It works best on the old version of a versioned model, but you can also set it on a model that has no versions at all. The value is a date, optionally with a time and a UTC offset. Supported forms are YYYY-MM-DD, YYYY-MM-DD hh:mm:ss.sss and YYYY-MM-DD hh:mm:ss.sss±hh:mm.

models:
  - name: my_model
    description: deprecating in the future
    deprecation_date: 2999-01-01 00:00:00.00+00:00

Leave out the offset, and the date is read in the time zone of whatever machine runs dbt. A laptop in Amsterdam and a CI runner set to UTC can then disagree about whether the date has passed. Include an offset if that matters to you.

For a model with an enforced contract, the date is binding as well as informative. An enforced contract (contract: enforced: true) is a set of guarantees about the model's structure that dbt checks on every build, and downstream consumers rely on it. Deleting the model would break exactly those consumers, so dbt will not let you delete it before its deprecation_date has passed. Deleting a versioned model early raises an error in development and fails jobs. The date is therefore the earliest point at which the old shape can be removed.

Sources123

2.Warnings producers and consumers see

A deprecation date only helps if the people still depending on the old shape hear about it. Once a date is set, dbt raises warnings that reach the consumer as well as the producer. Any project that references a deprecated or soon-to-be-deprecated model gets a warning at parse time. If the model is versioned and has a newer version, the warning says so. Three warning types cover the cases.

$ dbt parse
15:48:14  Running with dbt=1.6.0
15:48:14  Registered adapter: postgres=1.6.0
15:48:14  [WARNING]: While compiling 'my_model_ref': Found a reference to my_model, which is slated for deprecation on '2038-01-19T03:14:07-00:00'.

A consumer that must never depend on a dead model can make these warnings fail the run. WARN_ERROR_OPTIONS promotes them to runtime errors. Finding deprecated models takes a different approach, because no node selector exists for deprecation_date. List the property instead with dbt ls -q --output json --output-keys database schema alias deprecation_date and filter the output.

Sources1

3.Rolling out the new version

Replacing a shared model's shape without breaking anyone depends on two mechanics. latest_version decides which version an unpinned ref('fishtown_analytics_orders') resolves to; if you leave it out, the highest version number wins, so merely adding v2 would silently move every consumer onto it. A consumer pins a version by passing it in the ref, as in ref('fishtown_analytics_orders', v=1), and a ref with no v follows latest_version.

Retiring a version safely follows a fixed order, and the deprecation date goes on early. Suppose column_to_remove is being dropped from fishtown_analytics_orders. The producer:

1. Writes fishtown_analytics_orders_v2.sql and declares v1 and v2 in YAML, with latest_version: 1 so unpinned refs stay on v1 and consumers are not moved onto v2 by accident. 2. Sets a future deprecation_date on v1, merges, runs the job, and checks the run logs for v2's build and the deprecation warning. 3. Tells consumers that v2 exists and when v1 ends. 4. Changes latest_version to 2 once consumers have migrated, so unpinned refs now resolve to v2. 5. Cleans up v1.

Consumers have their own steps. When the warning appears, they test ref(..., v=2) against their own models. If v2 breaks something, they can pin v=1 for now, but the pin only lasts until the deprecation date. After the producer removes v1, a v=1 ref fails. Migrating means removing the version argument and fixing any downstream logic that depended on the old shape.

models:
  - name: fishtown_analytics_orders
    latest_version: 2 # update from 1 to 2 to set the new version as the latest version
    versions:
      - v: 1 # this represents the old version
      - v: 2 # this represents the new version

Sources4

4.Retiring the old version

Deprecation removes nothing by itself. A deprecated version keeps building, and consumers can still select it, until you disable or delete it. dbt does not drop the relation either, just as it leaves tables behind when a model is deleted. Until someone acts, the warehouse pays compute and storage for fishtown_analytics_orders_v1. On BigQuery, hours_to_expiration can expire the table automatically. dbt does not sync it with deprecation_date, so coordinate the two yourself, for example by expiring the table 48 hours after the date.

Removing a versioned model with an enforced contract takes two merges. The contract guarantees the model's columns to consumers, so dbt treats a vanishing model as a breaking change unless it has been deprecated first. Set a deprecation_date on the version being retired, using a past date if it is going right away, and run the production job so state-aware CI records the deprecation. Then remove or rename the file in a follow-up change. If you skip the first merge, [state comparison](lesson:7.1) can flag the removal as an unexpected breaking contract change and fail CI.

The cleanup steps below assume the naming convention. Each version lives in a file named <model_name>_v<v>, such as fishtown_analytics_orders_v1.sql, and by default materializes as a relation with the same name. The versions list in the model's YAML holds one entry per version (- v: 1, - v: 2), and those entries are the version specs. The base name, fishtown_analytics_orders without a suffix, is what BI tools and unpinned refs expect to find.

With either approach, BI tools still expect a relation under the base name. The latest_version_pointer config creates a view named after the base name, such as dim_customers, that points at the latest versioned relation. You can enable it per model or for the whole project with the latest_version_pointer_enabled_by_default flag. Then the next version bump moves dashboards automatically.

Sources4513

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Once a model's deprecation_date passes, dbt drops its table or view.Why is that wrong?

   dbt never drops relations for deprecated models. You have to drop them with a script, a cleanup macro or, on BigQuery, hours_to_expiration.

   Covered in Retiring the old version

2. 2.You can delete an old version with an enforced contract whenever its consumers say they have migrated.Why is that wrong?

   A model with an enforced contract cannot be deleted until its deprecation_date has passed. Deleting it early raises an error and fails jobs.

   Covered in Setting a deprecation_date

3. 3.You can select every deprecated model with a node selector such as --select deprecation_date:past.Why is that wrong?

   There is no selection syntax for deprecation_date. Use dbt ls with JSON output and filter on the property.

   Covered in Warnings producers and consumers see

Practise it for real

Watch deprecation warnings appear as a model moves from upcoming to past deprecation

1. 1.Pick a model in your project that at least one other model references with ref(), and add deprecation_date: 2999-01-01 to its YAML entry.

   Why: The reference warnings only fire when a project refers to the deprecated model.

   You should see: The YAML parses with no error.

2. 2.Run dbt parse.

   Why: Deprecation warnings are raised at parse time.

   You should see: A [WARNING] line saying the referencing model found a reference to your model, which is slated for deprecation on the date you set.

3. 3.Run dbt ls -q --output json --output-keys database schema alias deprecation_date.

   Why: No selector exists for deprecation_date, so you list the property instead.

   You should see: JSON lines, with your model's line showing the deprecation_date you set.

4. 4.Change the date to 1999-01-01 and run dbt parse again.

   Why: A past date moves the model from upcoming to deprecated.

   You should see: Warnings that the model is deprecated and that it is still being referenced, instead of the upcoming-deprecation warning.