What you will be able to do
- Name the automatic trigger types a Lakeflow job supports and what starts a run under each
- Explain why a Scheduled trigger is a poor fit when data arrives at irregular times or when latency matters
- Choose a file arrival or table update trigger based on where and how new data becomes available
Key concept
Data-driven trigger — A trigger that starts a job run because new data has actually landed, as files in a Unity Catalog location or as commits to a source table. A Scheduled trigger starts a run because the clock reached a set time. Choose a data-driven trigger when the availability of data should decide when work starts.
1.What can start a job run
Before a Lakeflow job does any work, something has to tell it to start. You set this in the Schedules & Triggers section of the Job details pane: click Add trigger and pick a trigger type. The types fall into two main groups. A time-based trigger (Scheduled) fires when the clock says so, whether or not new data has arrived. Data-driven triggers (File arrival, Table update, and Model update, which is in Beta) fire when something in Unity Catalog changes. Continuous is separate from both: it starts the next run as soon as the previous one ends. A job with no trigger still runs when someone clicks Run now or when an external orchestration tool starts it.
| Trigger type | Family | What starts a run |
|---|---|---|
| Scheduled | Time-based | A time-based schedule (simple interval or advanced cron-style schedule) |
| File arrival | Data-driven | New files arrive in a monitored Unity Catalog storage location |
| Table update | Data-driven | Source tables are updated |
| Model update | Data-driven (Beta) | A Unity Catalog model is created, a model version is ready, or a model alias is set |
| Continuous | Always on | The previous job run completes or fails |
| None (manual) | On demand | Run now button or other orchestration tools |
One setting applies to every trigger type. By default, only one run of a job can be active at a time, and runs that would exceed the configured max concurrency are skipped. So if a trigger fires more often than the job can finish, you don't get extra work done. You get skipped runs.
Checkpoint 1 of 6· Match them up
Match each trigger type to the event that starts a run
Tap a term, then the definition that fits it.
Only Scheduled is driven by the clock. File arrival and Table update react to data, and Continuous starts each run right after the last one.
“To keep the job always running, trigger another job run whenever a job run completes or fails.”Source: docs.databricks.com
Sources1
2.Time-based: what a Scheduled trigger does and does not promise
The Scheduled trigger has two modes. A simple schedule runs the job every N units of time, for example every twelve hours. You can't choose when the first run happens; the scheduler picks that time. An advanced schedule lets you set the period, starting time and time zone. You can also tick Show Cron Syntax to edit the schedule as a Quartz cron expression. Whatever the cron expression says, Databricks enforces at least 10 seconds between runs triggered by a schedule.
Two behaviours matter when you decide whether time is the right signal. First, the scheduler is not intended for low-latency jobs. Network or cloud issues can delay a run by several minutes, and the run then starts as soon as the service is available. Second, in a time zone that observes daylight saving time, an hourly job can be skipped, or appear an hour or two late, when the clocks change. Choose UTC if you need a run at every absolute hour.
A schedule also has no way to know whether the data is there. A 02:00 run fires at 02:00 whether the upstream export landed at 01:30 or will land at 04:00. That works when data arrives on a predictable cadence and consumers need output at a fixed time. It works poorly when arrival times vary.
Checkpoint 2 of 6· Check yourself
An hourly Scheduled job uses a time zone that observes daylight saving time, and the team notices a skipped run each spring. What change makes the job run at every hour in absolute time?
The skipped and late runs come from daylight saving changes. UTC has no daylight saving time, so an hourly schedule fires every absolute hour.
“To run at every hour (absolute time), choose UTC.”Source: docs.databricks.com
Sources2
3.Data-driven: file arrival triggers
When the data you depend on arrives as files, a file arrival trigger checks for the files themselves instead of guessing when they'll arrive. The documentation names the use case directly: the trigger is useful when irregular arrivals make a scheduled job inefficient. The trigger watches the root or a subpath of a Unity Catalog volume or external location, for example /Volumes/mycatalog/myschema/myvolume/mydirectory/, and recursively checks every subdirectory under it.
Detection isn't instant. The trigger makes a best effort to check for new files every minute, and the performance of the underlying cloud storage can affect this. For best performance, enable file events on the external location. Databricks then tracks new files using change notifications from the cloud provider. The trigger adds no cost beyond the cloud provider's charges for listing files.
The prerequisites are short, and the exam can ask about them: - Unity Catalog must be enabled. - The location must be a Unity Catalog volume or external location. - You need READ on the storage location and CAN MANAGE on the job.
Checkpoint 3 of 6· Exam question
A retail company runs a nightly reporting job that must always start at 2:00 AM regardless of whether new sales data has arrived, so downstream dashboards refresh on a predictable cadence. Which trigger should the data engineer configure for this Lakeflow Job?
Correct answer: B — Configure a scheduled trigger with a cron expression that fires the job at 2:00 AM every day, independent of upstream data changes.
- A. A file arrival trigger is data-driven and fires based on when new files land, so it would not reliably guarantee the job starts at exactly 2:00 AM every night.
- B. A cron-based scheduled trigger fires at a fixed wall-clock time regardless of whether upstream data has changed, which matches the requirement for a predictable 2:00 AM start every night.
- C. A table update trigger is data-driven and reacts to table changes, so run timing depends on when data lands rather than a fixed daily time.
- D. A continuous trigger restarts the job as soon as the previous run finishes, producing many runs per day rather than a single fixed nightly execution.
Checkpoint 4 of 6· Put it in order
Put the steps for adding a file arrival trigger to an existing job in order
- 1.Open the job and click Add trigger in the Job details pane
- 2.In Trigger type, select File arrival
- 3.Click Test connection to validate the configuration
- 4.In Storage location, enter the URL of a Unity Catalog volume or external location (root or subpath)
- 5.Click Save
You select the trigger type before you enter the storage location. You validate with Test connection before you save.
“To validate the configuration, click Test connection.”Source: docs.databricks.com
Checkpoint 5 of 6· Check yourself
A file arrival trigger monitors /Volumes/mycatalog/myschema/myvolume/mydirectory/. A file lands in mydirectory/subdirC/subdirD. What happens?
The trigger checks the configured path and every subdirectory under it, so one trigger on a parent directory covers nested landing folders.
“A file arrival trigger recursively checks for new files in all subdirectories of the configured location.”Source: docs.databricks.com
Sources3
4.Data-driven: table update triggers, and picking between the two
Often the data your job needs isn't a raw file but a table that another process maintains. A table update trigger watches one or more tables for data changes such as updates, merges and deletes, and starts a run when they change. Supported objects include: - Unity Catalog managed Delta and Iceberg tables - Unity Catalog external tables backed by Delta Lake - Materialized views and streaming tables - Views or metric views over supported tables
The main benefit is that your job doesn't need to know anything about the upstream process. The documentation describes running a job when new data is ready without knowing which processes update the table. If the upstream team changes how often they write, your job adjusts automatically.
To choose between the two data-driven triggers, ask where you can see that the data is ready. If readiness shows up as files landing in cloud storage, use file arrival. If it shows up as a commit to a Unity Catalog table, use table update. If neither signal exists and the business really needs output at a fixed time, a schedule is still the right tool.
Checkpoint 6 of 6· Exam question
A downstream Lakeflow Job needs to run its aggregation logic every time another team's managed Delta table, which the downstream team does not control or schedule, is updated with new rows. The downstream team wants the run to happen automatically without polling on a timer. Which trigger should they configure?
Correct answer: D — A table update trigger pointed at the upstream Delta table, so the job runs automatically whenever that table changes.
- A. A five-minute cron schedule polls on a fixed timer and can miss the exact moment of the update or run needlessly when nothing changed, which is less precise than reacting to the table itself.
- B. A file arrival trigger watches for new files in cloud storage, not row-level changes to a managed Delta table, so it does not directly observe table updates.
- C. A continuous trigger simply reruns the job as soon as the prior run finishes, which is unrelated to detecting when the specific upstream table changes.
- D. A table update trigger monitors a Unity Catalog managed table and starts a run whenever that table is updated, which directly matches reacting to another team's table without needing to know their schedule.
Sources4
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.A file arrival trigger reacts to a new file within seconds, like a push event.Why is that wrong?
It makes a best-effort check about once a minute, and slow cloud storage can delay that further. Enabling file events on the external location improves efficiency, but the documented check interval is still about one minute.
Covered in Data-driven: file arrival triggers
2.A tight cron schedule is a good way to get low-latency processing.Why is that wrong?
The job scheduler is not designed for low-latency work, and runs can be delayed by several minutes. A schedule also can't tell whether the data has arrived.
Covered in Time-based: what a Scheduled trigger does and does not promise
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://docs.databricks.com/aws/en/jobs/triggersOfficial docs
“Runs are skipped when they exceed the configured max concurrency for a job.”
↩︎ What can start a job run“Triggers a job run when new files arrive in a monitored Unity Catalog storage location.”
↩︎ What can start a job run“To keep the job always running, trigger another job run whenever a job run completes or fails.”
↩︎ Checkpoint - 2.https://docs.databricks.com/aws/en/jobs/scheduledOfficial docs
“You cannot specify the time for the first run for a simple schedule.”
↩︎ Time-based: what a Scheduled trigger does and does not promise“Databricks enforces a minimum interval of 10 seconds between subsequent runs triggered by the schedule of a job”
↩︎ Time-based: what a Scheduled trigger does and does not promise“The job scheduler is not intended for low-latency jobs.”
↩︎ Exam trap 2“job runs might occasionally be delayed up to several minutes”
↩︎ Prediction“To run at every hour (absolute time), choose UTC.”
↩︎ Checkpoint - 3.
“This feature is useful when a scheduled job's efficiency is compromised by irregular new data arrivals.”
↩︎ Data-driven: file arrival triggers“You must have READ permission on the storage location and CAN MANAGE permissions on the job.”
↩︎ Data-driven: file arrival triggers“File arrival triggers make a best effort to check for new files every minute”
↩︎ Exam trap 1“To validate the configuration, click Test connection.”
↩︎ Checkpoint“A file arrival trigger recursively checks for new files in all subdirectories of the configured location.”
↩︎ Checkpoint - 4.
“without the need for a continuously running cluster or knowledge of the processes that update a table”
↩︎ Data-driven: table update triggers, and picking between the two“run a job when new data is ready without the need for a continuously running cluster”
↩︎ Key concept