Running dbt models in sample mode with --sample

1.Why build on a slice of real data

The --sample flag saves development and CI time on large tables. A staging model over five years of events can take many minutes to build. Most of that time goes into reading history the developer doesn't need to check a new column or join. In sample mode, dbt filters what each model reads, so the model is built from a small but real subset of its inputs.

Sample mode goes one step past a [schema-only dry run with --empty](lesson:1.12). An empty run proves the SQL compiles and the DDL succeeds, but the resulting tables have zero rows, so there is nothing to inspect. A sampled run puts actual rows in your development schema. You can query the output and compare it with what you expect, which catches logic errors that an empty table would hide.

Sampling has a cost. Each input is cut down independently, so a fact table from the last three days may join to dimension rows that fall outside that window. Some joins will come back sparse or empty. A sampled build is good for validating logic quickly. It does not prove that the full build is correct.

Sources1

2.Relative and static time specs

Sampling is time-based only. The value you pass to --sample describes a time window, never a row count or a percentage. The window can take two forms. Pick the form based on what you are testing: recent behaviour, or a specific period in the past.

A relative spec moves with the clock, which makes it the usual choice during daily development. For an even larger model, set the window in hours, such as "6 hours". A static spec pins the window. That matters when the interesting data is in the past, such as last July's peak week, or when two people need to build against the same slice and compare results. The static example below mixes a plain date for the start with a full timestamp for the end, which is allowed.

dbt run --select path/to/stg_customers --sample="3 days"

dbt run --sample="{'start': '2024-07-01', 'end': '2024-07-08 18:00:00'}"

dbt run --select path/to/stg_customers --sample="6  ? "

Sources1

3.event_time tells dbt which column to filter

A time window is only useful if dbt knows which column holds the time. That column is set with the event_time config, and it is set on the upstream node: the model, seed, snapshot or source that is being read. The model you are building doesn't need it for its own inputs to be sampled. If stg_orders is the ref being filtered, stg_orders carries event_time.

Choose the column that records when the event happened, such as account_created_at or session_began_at. Avoid load timestamps like _fivetran_synced, which say when a row was ingested. The same config also drives [microbatch incremental models](lesson:1.14), so a well-chosen event_time serves both features. Sources are often the largest inputs, so they are where sampling saves the most.

sources:
  - name: source_name
    tables:
      - name: table_name
        config:
          event_time: event_timestamp

Sources21

4.Which commands and nodes are sampled

--sample is accepted by exactly two commands: dbt run and dbt build. That list is shorter than the one for --empty, which also works with snapshot and compile. Don't assume the two flags work in the same places.

Some node types behave differently inside a sampled run:

- Python models don't support the flag. dbt ignores it for them, so they read their full inputs. - Seeds are still created normally, with every row loaded. They are filtered only when a downstream node refs them, and seeds accept the event_time config like models, snapshots and sources, so set it on the seed to name its timestamp column. - A specific ref can be excluded from sampling by appending .render(), as in {{ ref('stg_customers').render() }}. Use this when a small lookup model must stay complete: sample the large fact inputs, render the small dimension in full, and the sampled facts still find their matching dimension rows.

Sources13

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.event_time only needs to be configured on the model you are running with --sample.Why is that wrong?

   Sampling filters the refs and sources that model reads, so each referenced node needs its own event_time naming its timestamp column.

   Covered in event_time tells dbt which column to filter

2. 2.--sample works with the same commands as --empty, including snapshot and compile.Why is that wrong?

   --sample is available only for run and build.

   Covered in Which commands and nodes are sampled

3. 3.A Python model run with --sample reads only the sampled window of its inputs.Why is that wrong?

   Python models don't support the flag. It is silently ignored, so they process their full inputs.

   Covered in Which commands and nodes are sampled

4. 4.--sample can take a row count or a percentage, such as 10% of the table.Why is that wrong?

   Only time-based sampling is supported, either as a relative window or as a static start and end.

   Covered in Relative and static time specs

Practise it for real

Build a staging model in your development schema from three days of sampled input data

1. 1.Add event_time to the source table that stg_customers reads, in its sources YAML under config, pointing at the column recording when each row's event occurred.

   Why: Sample mode filters refs and sources by the column named in event_time.

   You should see: dbt parses the project without errors.

2. 2.Run dbt run --select path/to/stg_customers --sample="3 days".

   Why: A relative spec filters the inputs back three days from now.

   You should see: stg_customers is built in your development schema, and the run is faster than a full build.

3. 3.Check the result. If stg_customers selects the event_time column, query its minimum and maximum in the built table. Otherwise, compare the table's row count with the source's row count.

   Why: This confirms the filter was applied to the input.

   You should see: Selected event_time values fall within roughly the last three days, or the row count is clearly lower than the full source.

4. 4.Rerun with a static spec, such as --sample="{'start': '2024-07-01', 'end': '2024-07-08 18:00:00'}".

   Why: A fixed window lets you test a known historical period reproducibly.

   You should see: The table now holds only rows from that window, if the source has data for it.