What you will be able to do
- Find and filter jobs in the Jobs & Pipelines list, including tag searches in key:value form
- Use the workspace-wide Runs tab, with its Finished runs count graph, Top 5 error types table and runs table, to spot failures
- Read a job's matrix view, including its colour codes, bar heights and expected-completion-time warnings
- Go from a failed run to the task run details page, and know how long run history is kept
- Use the active_only parameter of the runs list endpoint to separate active runs (PENDING, RUNNING, TERMINATING) from completed ones
Key concept
Matrix view (job run history) — On a single job's Runs tab, the matrix view is a grid. Each column is a run and each row is a task, and every cell is colour-coded by status. It lets you see at a glance which task failed in which run, and how long each run took.
1.Start at the Jobs & Pipelines list
All job monitoring in the UI starts from Jobs & Pipelines in the workspace sidebar. Its *Jobs & pipelines* tab shows every job and pipeline you have access to. For each one it gives a summary: who created it, what triggers it, and how its most recent runs ended. The last-five-runs indicator is the quickest health check you have. A job showing several recent failures is visible here before you open anything else. You can also choose which columns the list shows.
The unified Jobs & pipelines list is in Public Preview. Admins can switch it off and go back to the previous experience.
Once a workspace has hundreds of jobs, filtering becomes the main skill. The list supports:
- Text search on the Name and Job ID fields. Tags can be searched by key, by value, or by both together. - Type: jobs only, pipelines only, or all. - Owner: only the jobs or pipelines you own. - Favorites: items you have starred. - Tags: up to five tags at once from the drop-down. - Run as: up to two run-as identities.
Rows also have actions. The play button starts a job, the stop button stops it, and the kebab menu has more options, such as deleting the workflow.
Checkpoint 1 of 5· Check yourself
Your jobs carry a tag with key department and value finance. You want the list to match on both the key and the value. What do you type in the search box?
To match on both, enter the key and the value separated by a colon. Searching for the key alone or the value alone also works, but matches more broadly.
“To search by the key and value, enter the key and value separated by a colon (for example, department:finance).”Source: docs.databricks.com
Sources1
2.The Runs tab: every run across the workspace
The job list tells you what exists. The Runs tab next to it tells you what actually happened. It lists running and recently completed runs for every job and pipeline you can access. That includes runs started by external orchestrators such as Apache Airflow or Azure Data Factory. The tab has three parts, and each has its own scope:
| Component | What it shows | Scope |
|---|---|---|
| Finished runs count graph | Failed, skipped and successful runs by default; you can filter it to specific statuses | Runs completed in the last 48 hours |
| Top 5 error types | The most frequent error types | The selected time range |
| Runs table | Start time, name, type, run as, launched by, duration, status, error code, parameters | Runs from the last 60 days |
The filters across the top apply to all three parts. You can filter by name, type, pipeline type, Run as, start or end time, run status, Run ID and error code. The time filter has one detail worth knowing. For completed runs the range is matched against the run's end time, and for in-progress runs against its start time. You can also narrow the time range by dragging across the graph.
The graph also has visibility rules. It appears only when you filter to Jobs or to Pipelines. Admins see it for all runs, but non-admins must set Run as to *me*.
The status column uses a fixed set of values: Queued, Pending, Running, Skipped, Succeeded, Succeeded with failures, Failed, Timed Out, Canceling, or Canceled. If a run takes longer than the job's configured expected completion time, the duration column shows a warning.
Checkpoint 2 of 5· Check yourself
A non-admin opens the Runs tab with the type filter left on All, and the Finished runs count graph is missing. What brings it back?
The graph is hidden when All is selected. Non-admins also have to filter Run as to themselves before it shows their runs.
“The Finished runs count graph appears only when you filter to Jobs or Pipelines. It is not shown when All is selected.”Source: docs.databricks.com
Runs created with runs/submit are one-time runs. To find them, filter by Run ID, Run as, or Start time. If you want runs that show up in Name searches and support retries, create a saved job and run that instead of calling runs/submit.
The same split between running and finished work is available from the Jobs API. The list-runs endpoint (GET 2.0/jobs/runs/list) takes an active_only parameter. When active_only is true, only active runs are returned. When it is false or omitted, both active and completed runs are listed. An active run is one whose life_cycle_state is PENDING, RUNNING or TERMINATING. A run in any other life cycle state counts as completed. There is also a completed_only parameter, and the two cannot both be true. If you leave out job_id, the service lists runs from all jobs. The Databricks CLI exposes the same filters as databricks jobs list-runs --active-only and --completed-only.
Checkpoint 3 of 5· Exam question
A monitoring service polls the Jobs API every 30 seconds to build a live dashboard of jobs currently executing across a workspace with thousands of historical runs per job. Each poll should return only runs that have not yet reached a terminal state, without any client-side filtering. Which request against the List job runs endpoint achieves this?
Correct answer: A — Call `GET /api/2.2/jobs/runs/list` with `active_only=true` so the response returns only runs whose `life_cycle_state` has not yet reached a terminal value.
- A. The `active_only` flag has the server return only runs in a non-terminal `life_cycle_state`, so no completed runs are ever transferred to the dashboard. This is the correct, server-side filter for the scenario.
- B. `completed_only=true` does the opposite of what is needed by returning only terminated runs, and discarding successes locally still pulls the full history payload every poll.
- C. `run_type` filters by how a run was triggered, not by whether it is still executing, and sorting by `start_time` does nothing to exclude runs that have already finished.
- D. `expand_tasks` only adds per-task detail to whatever runs are already returned; it does not filter runs by state, and inferring the parent run's status from task fields is unreliable and adds unnecessary payload.
3.One job's history: the matrix view
When the workspace view points to a problem job, click the job's name. Its own Runs tab opens with a matrix view and a list view. In the matrix, the *Run total duration* row shows each run's total duration and state. The *Tasks* rows have one cell per task. Hovering over a task cell shows its start time, duration, cluster and status.
The matrix encodes two things visually. Colour shows status, and the height of each bar shows how long the run or task took. That makes a sudden slowdown easy to spot. If you have set an expected completion time, the matrix also shows a warning on any run that exceeded it.
Checkpoint 4 of 5· Match them up
Match each matrix-view colour to the status it represents
Tap a term, then the definition that fits it.
The colours come straight from the run bar legend. Note that skipped (pink) is a different colour from canceled (grey).
“Successful runs are green. Unsuccessful runs are red, skipped runs are pink, and waiting for retry is yellow.”Source: docs.databricks.com
Red and pink are different statuses: red marks an unsuccessful task and pink marks a skipped one. Neither always means the task's own code is broken. A task run may be unsuccessful because it failed, or it may have been skipped because a task it depends on failed. A pink cell in particular points upstream. To find the root cause, follow the dependency chain upstream.
The list view shows each run's start time, run ID, trigger (schedule, API request or manual), duration, status, error code and parameters. Active runs have a stop button. To stop everything at once, use Cancel runs or Cancel all queued runs from the drop-down. Go to the latest successful run takes you straight to the most recent good run, which is useful for comparison.
Answer: check the tasks it depends on. Pink means the task was skipped, and a task can be skipped because a dependent task failed. So look upstream for the red (failed) task and fix that first. The pink task's own code is probably not the problem.
4.Drilling into a run and its tasks
To open the job run details page, click the link in a run's Start time column. This page holds the job output and links to logs, and shows whether each task succeeded or failed. To get back to the Runs tab, click the Job ID value.
Jobs with multiple tasks offer three views of a run: a graph, a timeline and a list. In the graph, clicking a task node shows:
- the task's run as, launch method, start and end times, duration and status - the source code - the cluster that ran it, with links to its query history and logs - metrics for the task
The list view shows status, name, type, resource, duration and dependencies. You can search it by task name, filter it by status or type, and sort it by status, name or duration.
Checkpoint 5 of 5· Put it in order
You have been notified that a task failed. Put the UI steps for finding the cause in order.
- 1.Click Jobs & Pipelines in the sidebar
- 2.Click the failed task to open the Task run details page with its output and error message
- 3.In the Name column, click the job name to open its Runs tab
- 4.In the matrix view, hover over the failed task to see its metadata
The path narrows from the workspace, to the job, to the run history, to the single task. The Task run details page is where the error message and output appear.
“To help identify the cause of the failure, click the failed task. The Task run details page appears”Source: docs.databricks.com
There is a limit to keep in mind: Databricks keeps job run history for up to 60 days. After that the run, its output and its details are gone from the UI. If you need records for audits or longer trend analysis, export run results before they expire.
Sources1
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.Runs started from Airflow's DatabricksSubmitRunOperator can be found by filtering the Runs tab on a job name.Why is that wrong?
These are one-time runs/submit runs with no saved job behind them. Find them by Run ID, Run as or Start time instead.
Covered in The Runs tab: every run across the workspace
2.The Finished runs count graph covers the same 60-day window as the runs table.Why is that wrong?
The graph covers runs completed in the last 48 hours. Only the runs table goes back 60 days.
Covered in The Runs tab: every run across the workspace
3.Job run history stays in the Jobs UI indefinitely, so you can investigate a run from last quarter.Why is that wrong?
Run history is kept for up to 60 days. Export results first if you need them for longer.
Covered in Drilling into a run and its tasks
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://docs.databricks.com/aws/en/jobs/monitorOfficial docs
“lists information about all available jobs and pipelines, such as the creator, the trigger (if any), and the result of the last five runs.”
↩︎ Start at the Jobs & Pipelines list“Text search: keyword search is supported for the Name and Job ID fields.”
↩︎ Start at the Jobs & Pipelines list“including runs started by external orchestration tools such as Apache Airflow or Azure Data Factory”
↩︎ The Runs tab: every run across the workspace“The selected time range is matched against each run's end time for completed runs, or its start time for in-progress runs.”
↩︎ The Runs tab: every run across the workspace“Databricks retains run history for 60 days for both jobs and pipelines.”
↩︎ The Runs tab: every run across the workspace“For non-admins, you must click Run as and select me.”
↩︎ The Runs tab: every run across the workspace“The height of the individual job run and task run bars visually indicates the run duration.”
↩︎ One job's history: the matrix view“To stop all active and queued runs, select Cancel runs or Cancel all queued runs from the drop-down menu.”
↩︎ One job's history: the matrix view“The job run details page contains job output and links to logs”
↩︎ Drilling into a run and its tasks“Jobs with multiple tasks additionally have a graph, timeline, and list view.”
↩︎ Drilling into a run and its tasks“If you need to preserve job runs, Databricks recommends exporting results before they expire.”
↩︎ Drilling into a run and its tasks“The matrix view shows a history of runs for the job, including each job task.”
↩︎ Key concept“Because these runs have no associated job, you can't find them by filtering on a job Name.”
↩︎ Exam trap 1“The Finished runs count graph shows the number of runs completed in the last 48 hours.”
↩︎ Exam trap 2“Databricks maintains a history of your job runs for up to 60 days.”
↩︎ Exam trap 3“To search by the key and value, enter the key and value separated by a colon (for example, department:finance).”
↩︎ Checkpoint“The Finished runs count graph appears only when you filter to Jobs or Pipelines. It is not shown when All is selected.”
↩︎ Checkpoint“Because these runs have no associated job, you can't find them by filtering on a job Name.”
↩︎ Prediction“Successful runs are green. Unsuccessful runs are red, skipped runs are pink, and waiting for retry is yellow.”
↩︎ Checkpoint - 2.
“An active run is a run in the PENDING, RUNNING, or TERMINATING RunLifecycleState.”
↩︎ The Runs tab: every run across the workspace - 3.
“If active_only is true, only active runs are included in the results; otherwise, lists both active and completed runs.”
↩︎ The Runs tab: every run across the workspace - 4.
“A task run may be unsuccessful because it failed or was skipped because a dependent task failed.”
↩︎ One job's history: the matrix view“To help identify the cause of the failure, click the failed task. The Task run details page appears”
↩︎ Checkpoint