What you will be able to do
- Find the run history for one job and for the whole workspace, and say how far back each view goes
- Read the matrix view to spot runs that took longer than earlier runs, at both the job and the task level
- Set an expected completion time so the run history flags runs that go over your baseline
Key concept
Run history matrix view — The job-level Runs tab shows every recent run as a bar, and every task in each run as a cell. The height of each bar is the run's duration, so comparing bar heights is how you compare the current run with its history.
1.Two Runs tabs, two horizons
There are two places in Lakeflow Jobs to look at run history, and each one answers a different question. If you click Jobs & Pipelines in the sidebar and then the Runs tab, you get a workspace-wide list of the runs you can access. That includes runs started by external orchestrators such as Apache Airflow or Azure Data Factory. If you click a single job's Name instead, you get that job's own Runs tab. It has a matrix view and a list view of its active and completed runs. To see whether one job is getting slower, use the job-level tab. To see what is going wrong across the whole workspace, use the workspace-level tab.
The workspace view gives you a short-term pulse. The Finished runs count graph shows failed, skipped and successful runs over the last 48 hours. Next to it, a Top 5 error types table lists the most frequent errors in the time range you selected. The graph has some conditions. It appears only when you filter to Jobs or Pipelines, not when All is selected. Admins see it for all runs, but non-admins must set Run as to *me*. You can narrow the window with the time filter or by dragging across the graph. The filter compares the selected range with each completed run's *end* time, and with each in-progress run's *start* time.
The runs table underneath goes back further. Databricks keeps run history for 60 days for both jobs and pipelines. Each row shows how long the run took: the time elapsed so far for a running job, or the total running time for a completed one. The row also shows the status, what triggered the run, and any error code. Anything older than 60 days is gone from the UI. If you need to keep runs for longer, Databricks recommends exporting the results before they expire.
| View | How you open it | What it gives you for trend analysis |
|---|---|---|
| Workspace Runs tab | Jobs & Pipelines → Runs | Finished runs count graph (last 48 hours), Top 5 error types, and a table of runs from the last 60 days |
| Job Runs tab | Jobs & Pipelines → click the job's Name | Matrix view of run and task durations and states, plus a list view with each run's duration and status |
Checkpoint 1 of 5· Check yourself
A non-admin filters the workspace Runs tab to All and sees no Finished runs count graph. What is the most likely reason?
The graph is hidden when All is selected. For non-admins it also needs the Run as filter set to me.
“The Finished runs count graph appears only when you filter to Jobs or Pipelines.”Source: docs.databricks.com
Sources1
2.Reading trends in the matrix view
Open a job's Runs tab and the matrix view puts its recent runs next to each other. The top row, Run total duration, has one bar per run. Under it, each row of Tasks cells shows how one task did in each run. This layout is what lets you compare one run with the others. The bar height shows duration, so a run that took much longer than usual stands out as a taller bar among shorter ones. If you read along a task's row, you can tell whether the extra time came from one task or from all of them.
The colours tell you the outcome. Successful runs are green and unsuccessful runs are red. Skipped runs are pink, runs waiting for a retry are yellow, and pending, canceled or timed-out runs are grey. A tall green bar means the run succeeded but was slow. That is a performance trend, not a failure, and you would miss it if you only looked at status.
The bars show you the pattern. Hovering gives you the numbers. Hover over a bar in the Run total duration row to see that run's start time, duration and status. Hover over a task cell to see the task's start time, duration, cluster and status. The cluster detail is useful when a task got slower at the same point that its compute changed. To open the full details of a run, click the link in its Start time column in the list view. Go to the latest successful run takes you straight to the most recent green run, which is a handy baseline to compare against.
Checkpoint 2 of 5· Match them up
Match each part of the matrix view to what it tells you
Tap a term, then the definition that fits it.
Bar height shows duration. Hovering gives the details: the run bar shows start time, duration and status, and the task cell adds the cluster. Yellow means waiting for retry.
“To view details of the run, including the start time, duration, and status, hover over the bar in the Run total duration row.”Source: docs.databricks.com
Checkpoint 3 of 5· Exam question
In the Lakeflow Jobs Matrix view, the height of a colored bar in the "Run total duration" row and in each task row is scaled to represent which quantity for that run or task?
Correct answer: A — The bar height encodes how long that run or task took to complete, so a taller bar always represents a longer measured duration.
- A. This is correct because the Matrix view scales each cell's bar height to the measured run or task duration, letting engineers scan a row visually to spot which executions ran long. Color separately encodes status, while height is reserved for the time dimension.
- B. This is incorrect because the Matrix view does not surface billed compute cost as a visual dimension; cost is not one of the metrics the run history view renders per cell. Bar height in this view is tied to duration, not spend.
- C. This is incorrect because retry counts are not represented through bar height; a run waiting for retry is instead shown through the yellow status color, not through a taller or shorter bar. Height stays tied to elapsed duration regardless of retry activity.
- D. This is incorrect because the number of downstream tasks in a job's dependency graph is a structural property shown by the task rows themselves, not by scaling any single cell's bar height. Height in the Matrix view reflects time elapsed, not graph fan-out.
Sources1
3.Turning a baseline into a warning
Comparing bar heights by eye works, but the baseline exists only in your head. To make it explicit, open the Job details pane and click Duration and streaming backlog thresholds. Then choose Run duration as the metric. The setting has two fields that do different jobs:
- Warning is the job's *expected completion time*. When a run goes over it, an event is triggered, and both the matrix view and the runs list show a warning on that run. - Timeout is the *maximum completion time*. A run that doesn't finish within it ends with the status Timed Out.
So Warning tells you a run is slower than normal and lets it keep going. Timeout stops it. The Warning event can also send you an alert. When you add or edit a notification, choose Duration Warning to get an email or system notification when the job or task goes over the threshold. Then you hear about slow runs as they happen and don't have to spot them in the matrix later.
Checkpoint 4 of 5· Check yourself
You want to be alerted when a nightly job runs longer than its usual 30 minutes, but you don't want the run to stop. What should you configure?
The Warning field sets the expected completion time, and going over it triggers an event that a Duration Warning notification can send to you. The Timeout field would end the run as Timed Out.
“Enter a duration in the Warning field to configure the job's expected completion time.”Source: docs.databricks.com
Checkpoint 5 of 5· Exam question
A nightly Lakeflow job that normally finishes in about 10 minutes took 45 minutes last night, and no code or configuration changed. What is the best way to determine whether this is a genuine performance regression rather than simply a larger data volume that night?
Correct answer: A — Open the slow run alongside a recent fast run and compare rows read and written plus total query counts per task to see if workload size grew.
- A. This is correct because comparing rows read and written and per-task query counts between the slow run and a recent fast run reveals whether the extra time is explained by more data flowing through the pipeline or by a genuine slowdown at the same workload size. This side-by-side diagnostic is the recommended way to separate a workload-shape change from a real regression.
- B. This is incorrect because removing the expected completion time setting only stops future warnings from appearing; it does not explain why last night's specific run took four times longer and provides no diagnostic evidence about data volume or regression.
- C. This is incorrect because changing the cluster type affects startup and warm-up behavior going forward but does not, by itself, tell you whether last night's slowdown came from more data or a genuine regression. It changes future runs rather than diagnosing the one that already happened.
- D. This is incorrect because adding retries addresses transient task failures, not diagnosis of a slow-but-successful run, and it does nothing to distinguish a data volume increase from an actual performance regression.
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.The run history automatically flags runs that are slower than the job's historical average.Why is that wrong?
The matrix view and runs list show a duration warning only after you have configured an expected completion time (the Warning field). Without it, the only sign of a slow run is a taller bar.
Covered in Turning a baseline into a warning
2.Setting a duration threshold in the Timeout field is how you get alerted about slow runs.Why is that wrong?
Timeout is the maximum completion time: a run that goes over it ends as Timed Out. To flag a slow run and let it keep going, use the Warning field (the expected completion time) with a Duration Warning notification.
Covered in Turning a baseline into a warning
3.The Finished runs count graph shows the same 60-day window as the runs table.Why is that wrong?
By default the graph covers runs completed in the last 48 hours. The runs table holds the 60 days of history that Databricks keeps.
Covered in Two Runs tabs, two horizons
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://docs.databricks.com/aws/en/jobs/monitorOfficial docs
“The selected time range is matched against each run's end time for completed runs, or its start time for in-progress runs.”
↩︎ Two Runs tabs, two horizons“If you need to preserve job runs, Databricks recommends exporting results before they expire.”
↩︎ Two Runs tabs, two horizons“The matrix view shows a history of runs for the job, including each job task.”
↩︎ Reading trends in the matrix view“Successful runs are green. Unsuccessful runs are red, skipped runs are pink, and waiting for retry is yellow.”
↩︎ Reading trends in the matrix view“The height of the individual job run and task run bars visually indicates the run duration.”
↩︎ Key concept“If you have configured an expected completion time, the matrix view displays a warning when the duration of a run exceeds the configured time.”
↩︎ Exam trap 1“Databricks retains run history for 60 days for both jobs and pipelines.”
↩︎ Exam trap 3“The Finished runs count graph shows the number of runs completed in the last 48 hours.”
↩︎ Prediction“The Finished runs count graph appears only when you filter to Jobs or Pipelines.”
↩︎ Checkpoint“To view details of the run, including the start time, duration, and status, hover over the bar in the Run total duration row.”
↩︎ Checkpoint“If you have configured an expected completion time, the matrix view displays a warning when the duration of a run exceeds the configured time.”
↩︎ Prediction - 2.
“If the job exceeds this threshold, an event is triggered.”
↩︎ Turning a baseline into a warning“To configure a maximum completion time for a job, enter the maximum duration in the Timeout field.”
↩︎ Exam trap 2“Enter a duration in the Warning field to configure the job's expected completion time.”
↩︎ Checkpoint - 3.
“To receive a notification for a job or task that exceeds a duration threshold, select Duration Warning when you add or edit a notification.”
↩︎ Turning a baseline into a warning