What you will be able to do
- Name the trigger types a Lakeflow job can use and say what starts a run under each
- Configure a simple or advanced (Quartz cron) scheduled trigger and predict how time zones affect it
- Explain how continuous mode differs from a schedule and what it rules out
- Predict what happens to extra runs under the default concurrency limit and when a trigger is paused
Key concept
Trigger — A trigger is the job-level setting that decides when a job runs. It can follow the clock (a schedule) or react to an event such as new data. A job with no trigger runs only when someone starts it.
1.What can start a job run
A Lakeflow job is made of tasks, but the job as a whole decides when it runs, and the trigger is the setting that does that. A new job has no trigger. Out of the box it runs only when you start it by hand with Run now or from an outside orchestration tool. To make it run automatically, open the job and go to the Schedules & Triggers section of the Job details pane. Click Add trigger and choose a type. When you click Save, the job starts running on that trigger.
Triggers come in two broad kinds. A time-based trigger follows the clock, for example every day at 2 AM. An event-based trigger reacts to something, such as new data arriving in cloud storage. The exam objective covers three types by name: scheduled, file arrival and table update. Continuous is a fourth type you need to tell apart from a schedule. Model update also exists, but it is in Beta and outside this objective.
| Trigger type | What starts a run | Time-based or data-driven |
|---|---|---|
| Scheduled | A time-based schedule (simple interval or advanced/cron) | Time-based |
| File arrival | New files arriving in a monitored Unity Catalog storage location | Data-driven |
| Table update | Updates to one or more monitored source tables | Data-driven |
| Continuous | Another run starts whenever a run completes or fails | Always running |
| None (manual) | Run now button or other orchestration tools | Manual |
Checkpoint 1 of 5· Match them up
Match each trigger type to what starts a job run
Tap a term, then the definition that fits it.
Each trigger type has one condition that starts a run. Scheduled follows the clock, file arrival and table update react to data, and continuous starts the next run as soon as the last one ends.
“Triggers a job run when new files arrive in a monitored Unity Catalog storage location.”Source: docs.databricks.com
2.Scheduled triggers: simple intervals and Quartz cron
The Scheduled trigger has two schedule types.
- Simple: you set an interval and a unit of time, such as every twelve hours. You cannot choose when the first run happens. The scheduler picks that time when you configure the schedule, and the interval counts from the job's first run. - Advanced: you set the period, the starting time and the time zone. If you tick Show Cron Syntax, you can view and edit the schedule as a Quartz cron expression.
If an exam scenario needs a run at a specific time of day, the answer is an advanced schedule, not a simple one.
Checkpoint 2 of 5· Put it in order
Put the steps for adding a scheduled trigger to an existing job in order
- 1.In the Job details pane, click Add trigger
- 2.In Schedule type, select Simple or Advanced and set the schedule
- 3.In the workspace sidebar, click Jobs & Pipelines
- 4.In Trigger type, select Scheduled
- 5.Click your job's Name link
- 6.Click Save
Open the job from Jobs & Pipelines, add a trigger from the Job details pane, choose Scheduled, set the schedule type, then save.
“In Schedule type, select Simple or Advanced. Simple: Specify an interval and unit of time to run the job.”Source: docs.databricks.com
Three behaviours of the scheduler are easy to miss:
1. Minimum interval. Runs started by a schedule are always at least 10 seconds apart, even if the cron expression asks for something more frequent. 2. Time zones. You can pick a time zone that observes daylight saving time, or UTC. With a daylight-saving zone, an hourly job may be skipped, or appear an hour or two late, when daylight saving starts or ends. To run every hour in absolute time, use UTC. 3. Latency. The scheduler is not meant for low-latency jobs. Network or cloud problems can occasionally delay a run by up to several minutes. When the service is available again, the delayed scheduled jobs run straight away.
Checkpoint 3 of 5· Check yourself
An hourly scheduled job must run every hour in absolute time, without being skipped or shifted when daylight saving time begins or ends. What should you configure?
Daylight-saving time zones can skip or shift hourly runs at the changeover. The docs say to choose UTC to run every hour in absolute time.
“To run at every hour (absolute time), choose UTC.”Source: docs.databricks.com
Sources4
3.Continuous mode: always running, with fewer options
A schedule starts runs at set times. Continuous mode starts a new run whenever the previous one completes or fails, so the job never stops. Databricks recommends it for always-on streaming workloads. It replaces the older advice to give streaming jobs an unlimited retry policy and a maximum of one concurrent run.
Continuous mode has some constraints:
- Only one instance of a continuous job can run at a time. - There is a short gap between one run finishing and the next one starting, which should be under 60 seconds. - You cannot use task dependencies or retry policies. If the job fails, the whole job is retried with exponential backoff. You can also turn on task-level retries by setting Task retry mode to On failure. - To stop the job, click Pause. To start it again in continuous mode, click Resume.
If you change a continuous job's configuration, the change takes effect only after you click Restart run.
Checkpoint 4 of 5· Check yourself
Which statement about a job configured with a Continuous trigger is correct?
Continuous mode allows only one running instance. It doesn't support task dependencies, and it isn't driven by cron. A new run starts when the previous one ends.
“There can be only one running instance of a continuous job.”Source: docs.databricks.com
Sources5
4.Concurrent runs and pausing a trigger
By default, only one run of a job can be active at a time. You can raise this limit under Advanced settings (maximum concurrent runs). If a trigger fires while the job is already at its limit, the new run is skipped. It is not queued to run later. In the scenario above, the runs that come due during the long run are skipped, and the next run happens at the first scheduled time after the long run has finished.
Once a job has a trigger, you can edit, pause, resume or delete it from the Schedules & Triggers section of the Job details pane:
- Edit trigger changes the configuration. - Pause stops the trigger from starting new runs. Runs that are already active keep going. - Resume brings back the behaviour you configured, on the same schedule as before. - Delete removes the trigger from the job.
When you create or edit a trigger, you can also set its Trigger Status to Active or Paused directly. For a continuous trigger, if a run is still active when you resume it, the scheduler waits for that run to finish before it starts a new one.
Checkpoint 5 of 5· Exam question
A support team receives a steady flood of small log files that arrive continuously all day into a monitored volume. Every arrival is a valid trigger event, but the downstream cluster is expensive to start, and the team wants the job to run at most once every 30 minutes even if files keep arriving faster than that. Which configuration achieves this without abandoning event-driven triggering?
Correct answer: A — Set `min_time_between_triggers_seconds` to 1800 on the file arrival trigger so that, after a run finishes, further arrivals cannot start a new run until that cooldown elapses.
- A. A 1800-second cooldown after each completed run caps how often a new run can start, which is exactly the once-every-30-minutes cadence the team wants while still reacting to file arrivals rather than a fixed schedule.
- B. Because log files arrive continuously all day, a wait-after-last-change timer would keep getting reset by new files and could delay the first run indefinitely, which is the opposite of the steady 30-minute cadence needed here.
- C. Dropping automated triggering in favor of manual runs removes the event-driven behavior the team wants to keep and adds an ongoing operational burden of someone starting runs by hand.
- D. Limiting concurrent runs prevents overlapping executions but says nothing about the minimum time between separate, non-overlapping runs, so frequent arrivals could still start a new run every few minutes.
No. The active run keeps going, and the trigger simply stops starting new runs. When you click Resume, the trigger picks up again on the schedule that was configured before.
Sources2
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.When a trigger fires while the job is already running, the new run is queued and starts once the current run finishes.Why is that wrong?
The default limit is one active run. A run that would go over the configured max concurrency is skipped, not queued.
Covered in Concurrent runs and pausing a trigger
2.A simple schedule lets you choose the exact time of the first run.Why is that wrong?
A simple schedule only takes an interval and a unit. The scheduler chooses when the first run happens. To control the start time, use an advanced schedule.
Covered in Scheduled triggers: simple intervals and Quartz cron
3.A continuous job can keep its task dependencies and job retry policy.Why is that wrong?
Continuous jobs can't use task dependencies or retry policies. Instead, they retry the entire job with exponential backoff.
Covered in Continuous mode: always running, with fewer options
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“By default, it only runs when you manually start it, but you can also configure it to run automatically.”
↩︎ What can start a job run - 2.https://docs.databricks.com/aws/en/jobs/triggersOfficial docs
“In the Job details pane, scroll down to the Schedules & Triggers section, and then click Add trigger.”
↩︎ What can start a job run“By default, only a single run of a job can be active at a time.”
↩︎ Concurrent runs and pausing a trigger“Any currently active runs continue, but the trigger no longer starts new runs.”
↩︎ Concurrent runs and pausing a trigger“Runs are skipped when they exceed the configured max concurrency for a job.”
↩︎ Exam trap 1“Triggers a job run when new files arrive in a monitored Unity Catalog storage location.”
↩︎ Checkpoint - 3.https://docs.databricks.com/aws/en/jobsOfficial docs
“or event-based, such as running a job when new data arrives in cloud storage.”
↩︎ What can start a job run“A trigger is a mechanism that initiates running a job based on specific conditions or events.”
↩︎ Key concept - 4.https://docs.databricks.com/aws/en/jobs/scheduledOfficial docs
“Optionally select the Show Cron Syntax checkbox to display and edit the schedule in Quartz Cron Syntax.”
↩︎ Scheduled triggers: simple intervals and Quartz cron“Databricks enforces a minimum interval of 10 seconds between subsequent runs triggered by the schedule of a job”
↩︎ Scheduled triggers: simple intervals and Quartz cron“The job scheduler is not intended for low-latency jobs.”
↩︎ Scheduled triggers: simple intervals and Quartz cron“You cannot specify the time for the first run for a simple schedule.”
↩︎ Exam trap 2“In Schedule type, select Simple or Advanced. Simple: Specify an interval and unit of time to run the job.”
↩︎ Checkpoint“To run at every hour (absolute time), choose UTC.”
↩︎ Checkpoint - 5.https://docs.databricks.com/aws/en/jobs/continuousOfficial docs
“Databricks recommends using continuous mode for always-on streaming workloads.”
↩︎ Continuous mode: always running, with fewer options“A delay exists between a run finishing and a new run starting. This delay should be less than 60 seconds.”
↩︎ Continuous mode: always running, with fewer options“You cannot use task dependencies with a continuous job.”
↩︎ Exam trap 3“continuous jobs automatically retry the entire job on failure using an exponential backoff algorithm.”
↩︎ Prediction“There can be only one running instance of a continuous job.”
↩︎ Checkpoint