What you will be able to do
- Explain why a context failure can only be diagnosed from the intermediate steps of a trace, not from the final answer alone
- Recognise the trace signature of context poisoning, context distraction, context confusion and context clash
- Tell the four modes apart when they produce similar wrong answers, and name the remediation each one points to
- Place diagnosis inside the find, measure, fix, confirm, monitor loop
Key concept
Context failure mode — A context failure mode is a recurring way an agent goes wrong because of what is in its context window, not because the model lacks capability. You diagnose it by finding the step where the bad context came in and following how later steps used it.
1.The context is the evidence, and the trace is where you read it
An agent runs in a loop. On each step the model reads its whole context: the system prompt, the task, any retrieved documents, recalled memory, and the results of earlier tool calls. Then it picks the next action. The harness runs that action and adds the result back into the context. The model never sees anything else. So when an agent fails, the first question to ask is what the model was actually looking at when it made the bad decision, and only then what is wrong with the model.
That is why diagnosis starts with a trace. On Databricks, an MLflow trace records one complete execution of the agent as a tree of spans: LLM calls, retriever lookups, tool invocations, and the inputs, outputs, latency, token usage and errors of each one. The final answer tells you *that* something went wrong. The spans tell you *where* bad or excess context came in and *how* later steps used it. Genie Code's trace analysis is scoped the same way. It points you to the span or step where a failure happened, not to a line of source code.
Checkpoint 1 of 7· Check yourself
An agent returns a wrong final answer. Why can't you tell which kind of context failure caused it from the input and final response alone?
Input and output alone hide the intermediate steps. A trace captures each model call, retrieval and tool call, so you can see which step introduced or misused bad context.
“With only an input and final response, you cannot tell which intermediate decision caused a failure.”Source: docs.databricks.com
The exam uses four labels for context-driven failures. Poisoning: a wrong fact gets into the context and is reused. Distraction: there is so much context that it crowds out the model's own reasoning. Confusion: irrelevant material in the context gets used. Clash: two parts of the context contradict each other. Each one leaves a different mark on a trace. The sections below take them one at a time.
2.Context poisoning: one bad fact, reused
Context poisoning happens when an error gets into the context and the agent keeps treating it as true. The source can be a hallucination in an earlier model step, such as a made-up order ID, a wrong assumption written into a plan, or a misread tool error. It can also come from outside: a stale, wrong, or deliberately planted document returned by retrieval. Databricks' RAG guidance warns that document sources you feed an agent are possible attack vectors, so retrieval is one of the ways poison gets in.
Trace signature. Look for one span where a wrong value first appears. Every later span then builds on it, so the error spreads forward. The agent might call tools with the hallucinated ID, retry the same impossible goal, or write a plan that rests on the false premise. The wrong value shows up consistently, not randomly. The later reasoning is often internally coherent, just anchored to a wrong starting point. If you follow the bad value back to its earliest span, you have found where the poisoning started.
Poisoning is worse when the bad fact gets written to durable memory. Databricks keeps session state (one interaction's transcript) separate from memory (facts recalled in later, unrelated conversations). A poisoned fact distilled into memory will come back in future sessions even after the original conversation is gone. The fix is to remove the bad content at its source: correct or delete the memory entry, filter the retrieved document, or validate tool outputs and model claims before they are allowed to stay in context.
Checkpoint 2 of 7· Check yourself
An agent once hallucinated a user's account tier and saved it as a durable fact. The team clears that user's session store, but the wrong tier keeps showing up in new conversations. What explains this?
Sessions and memories are separate stores. A poisoned fact that was written to memory survives the deletion of any session, so you have to correct the memory entry itself.
“deleting a session, or an entire session store, does not delete memory retained in a memory store.”Source: docs.databricks.com
3.Context distraction: so much context the model stops thinking
Context distraction is a volume problem. Nothing in the context has to be false. There is simply so much of it, usually a long, growing history of turns, tool outputs, and sub-agent logs, that the model pays too much attention to the accumulated context and not enough to its own reasoning. A distracted agent tends to repeat actions from its history instead of planning fresh ones. It also loses track of the original goal, or mishandles instructions that are now buried in the middle of a very long prompt.
Trace signature. Input tokens per LLM span keep climbing over the run. You can read token usage per span and for the whole trace. Quality drops as the trace gets longer even though no single span added a wrong fact. The agent loops back over earlier tool calls or copies the shape of earlier answers. Pasting large raw outputs into the context, such as full tool logs or a sub-agent's complete reasoning, is a common trigger. The usual remedy is to cut volume, not to remove a specific bad fact. Harnesses trim or summarise older context so the model is not overwhelmed. That technique belongs to a neighbouring lesson. Here, you only need to recognise when it's the right fix.
Checkpoint 3 of 7· Exam question
In a long-running coding agent session, a large raw file listing (thousands of lines) returned by a tool call several turns ago is still sitting in the message history, even though the agent has since moved on to unrelated edits. What is the most effective way to reduce the attention this stale content consumes?
Correct answer: A — Apply tool result clearing to discard the superseded raw output while keeping the turn's outcome in history
- A. Correct. Once a tool result deep in history is no longer needed verbatim, clearing that raw output is one of the lightest-touch and safest forms of compaction, freeing attention budget while preserving the decision or outcome that mattered.
- B. Incorrect. A higher reasoning effort spends more compute per response but does not remove the stale tokens still occupying context, so the underlying dilution remains.
- C. Incorrect. Re-running the tool call adds another large, mostly redundant listing to history, increasing token consumption rather than reducing it.
- D. Incorrect. An added instruction competes for attention just like any other token and does not actually remove the large stale listing from context, so the disproportionate consumption remains.
4.Context confusion: irrelevant material gets used
Context confusion happens when content that is in the context but shouldn't matter ends up shaping the response. The classic case is tool overload. Give a model dozens of similar tool definitions and it will sometimes call the wrong one, or call a tool when none was needed, just because the tool was offered. The same thing happens with retrieved chunks that look on-topic but are irrelevant, or with overlapping data sources that describe the same concept in slightly different ways.
Databricks' Genie Agent troubleshooting describes the same mechanism in a text-to-SQL setting. When tables overlap in columns or concepts, the agent has trouble telling which data to use. The recommended fix is to remove the extra options, not to add more instructions. As a general design point, an agent that has to cover radically different sub-domains can become overloaded with functionality.
Trace signature. The wrong span is a *selection*: a tool call that doesn't fit the request, a query against the wrong table, or an answer that relies on a retrieved chunk unrelated to the question. Unlike poisoning, nothing in the context is false. The tool or document is valid, just irrelevant here. Unlike distraction, the context doesn't need to be long. Even a short context can confuse the model if it offers too many similar options.
Checkpoint 4 of 7· Check yourself
A Genie Agent keeps querying the wrong one of two tables that both contain a 'revenue' column with slightly different definitions. Which change most directly addresses the cause?
The failure comes from overlapping options in context. Removing the redundant table or columns takes away the source of confusion. More history, a bigger window, or an extra checker leaves the overlapping options in place.
“If possible, remove unnecessary or overlapping tables or columns.”Source: docs.databricks.com
Checkpoint 5 of 7· Exam question
A context engineer must choose a reasoning-effort setting for an agent built on a reasoning-only model family (such as the GPT-5 family) available through Databricks Foundation Model APIs. Which parameter and value range are used to control this?
Correct answer: A — `reasoning_effort`, with values such as `"low"`, `"medium"`, or `"high"`
- A. Correct. Reasoning-only models exposed through Databricks Foundation Model APIs, such as the GPT-5 family, use a `reasoning_effort` parameter with values like low, medium, or high to control how much internal deliberation the model performs.
- B. Incorrect. `budget_tokens` is the parameter used for hybrid reasoning models like Claude and Gemini 2.5 to size a thinking budget, and it must stay below (not exceed) `max_tokens`; it is not how reasoning-only model effort is configured.
- C. Incorrect. `thinking` with `{"type": "enabled"}` is the hybrid-reasoning toggle used by Claude-style models, not the effort control for reasoning-only model families.
- D. Incorrect. `attention_budget` with narrow/balanced/wide values is not a documented Databricks Foundation Model APIs parameter; it does not correspond to any real reasoning-mode control.
5.Context clash: the context disagrees with itself
Context clash happens when two parts of the context contradict each other and the model has no reliable way to pick a winner. This is common in multi-turn work. Session state is usually an append-only transcript of messages, tool calls, and results that the agent re-reads every turn. Early guesses made before the facts arrived therefore stay in the context next to the later corrections. A user who gives requirements piece by piece, a tool that returns a newer value than an earlier tool, or two retrieved policies with different dates all produce the same result: contradictory statements sitting side by side.
Trace signature. Search the inputs of the failing LLM span for two incompatible statements about the same thing. The output often hedges, flips between turns, or follows the earlier (wrong) statement because it came first and the model's previous reasoning already committed to it. What separates clash from poisoning is that the correct information *is* in the context. The trouble is that it's competing with something else. Fixes remove the competition: prune or replace outdated turns, consolidate the requirements into one restated prompt, or mark which source takes priority.
Clash. The correct 14-day value is in the context, but it's competing with the stale 30-day document. If only the stale document had been retrieved and then reused, it would be poisoning. Fix it by giving the authoritative source priority or by filtering stale documents out of retrieval.
Checkpoint 6 of 7· Exam question
A team is deciding on a reasoning mode for a routing agent that only classifies incoming tickets into one of five known categories before handing off to a specialist agent. Token cost per request is a primary concern because volume is extremely high. What is the strongest justification for using reduced or standard reasoning here?
Correct answer: A — The classification task is narrow and low-ambiguity. Extra reasoning tokens would raise per-request cost at high volume without materially improving classification accuracy.
- A. Correct. The classification task is narrow and low-ambiguity, so extra reasoning tokens would increase cost without meaningfully improving accuracy. At high volume, this cost multiplies, making reduced or standard reasoning the optimal token-budget tradeoff.
- B. Incorrect. The choice of reasoning mode is dictated by the complexity and cost requirements of the task itself, not by whether the agent hands off to another agent. Handoffs do not mandate reduced reasoning; a more complex handoff task could warrant extended thinking.
- C. Incorrect. Extended thinking is a per-call configuration and is not universally forbidden in multi-agent systems, including Databricks orchestration. While cost- and latency-sensitive classification tasks may use reduced or standard modes, the platform does not categorically prohibit extended reasoning.
- D. Incorrect. Reasoning mode selection determines the number of tokens spent on deliberation, not the size of the context window. Standard reasoning does not inherently provide a larger context window than extended thinking, so it does not better retain full ticket histories.
Sources5
6.From diagnosis to a fix that stays fixed
Telling the four modes apart comes down to a few questions about the trace. Is there one span where a wrong value first appears and then spreads? That's poisoning. Does quality drop as the context grows, with no single bad input? That's distraction. Did the agent pick an irrelevant tool, table, or chunk that was simply there to be picked? That's confusion. Are there two contradictory statements in the same context? That's clash. Each answer points to a different fix, which is why getting the label right matters more than spotting that the answer was wrong.
Diagnosing one trace is only the start. Databricks frames agent quality as a repeating loop. You find an issue in traces, often with Genie Code questions such as finding traces where the retriever returned nothing but the agent answered anyway. Then you measure it across many traces with a scorer, fix the context design, confirm with evaluation that the fix worked and didn't break anything else, and monitor production so the failure doesn't come back.
Checkpoint 7 of 7· Put it in order
Put the stages of the agent quality loop in order, starting from a context failure you just spotted in a trace.
- 1.Fix it
- 2.Measure it systematically
- 3.Confirm the fix
- 4.Find the issue in your agent's traces
- 5.Monitor so it does not recur
Databricks describes observability as one repeating loop: find, measure, fix, confirm, monitor. Monitoring then surfaces the next issue.
“find an issue in your agent, measure it systematically, fix it, confirm the fix, and monitor so it doesn't recur.”Source: docs.databricks.com
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.Clearing or restarting the conversation always gets rid of a poisoned fact.Why is that wrong?
If the bad fact was written to durable memory, it lives outside the session and will come back in new conversations until the memory entry itself is fixed.
Covered in Context poisoning: one bad fact, reused
2.Distraction is solved by giving the model a larger context window so everything fits.Why is that wrong?
Distraction comes from the volume of context, not a lack of space. The fix is to keep distracting material out and trim what has built up, not to let in more.
Covered in Context distraction: so much context the model stops thinking
3.Giving an agent more tools or data sources can only help, because it will ignore what it doesn't need.Why is that wrong?
Options that overlap or don't apply are themselves a cause of wrong selections. The documented fix is to remove them.
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“A trace is a structured record of one complete execution of your agent”
↩︎ The context is the evidence, and the trace is where you read it“find an issue in your agent, measure it systematically, fix it, confirm the fix, and monitor so it doesn't recur.”
↩︎ From diagnosis to a fix that stays fixed - 2.
“means identifying the span, input pattern, or execution step in the traces where a failure occurred”
↩︎ The context is the evidence, and the trace is where you read it“Find traces where the retriever returned no results but the agent still tried to answer”
↩︎ From diagnosis to a fix that stays fixed - 3.https://www.databricks.com/blog/ai-harnessSecondary source
“The model reads everything in its context, including the task, relevant memory and previous results, then decides what action to take next.”
↩︎ The context is the evidence, and the trace is where you read it“In practice, this means trimming older parts of the conversation so the model does not become overwhelmed as the context grows.”
↩︎ Context distraction: so much context the model stops thinking - 4.https://docs.databricks.com/aws/en/agents/tutorials/ai-cookbook/quality-data-pipeline-ragOfficial docs
“any document sources you feed into your agent are potential attack vectors for bad actors to launch data poisoning attacks.”
↩︎ Context poisoning: one bad fact, reused“minimizing the inclusion of distracting or irrelevant information”
↩︎ Context distraction: so much context the model stops thinking“minimizing the inclusion of distracting or irrelevant information”
↩︎ Exam trap 2 - 5.
“Memories are scoped to a subject, not an interaction”
↩︎ Context poisoning: one bad fact, reused“Most commonly this is the conversation history (the ordered transcript of messages, tool calls, and results)”
↩︎ Context clash: the context disagrees with itself“The agent reads it at the start of a turn and appends to it as the interaction runs.”
↩︎ Context clash: the context disagrees with itself“deleting a session, or an entire session store, does not delete memory retained in a memory store.”
↩︎ Exam trap 1“deleting a session, or an entire session store, does not delete memory retained in a memory store.”
↩︎ Checkpoint - 6.
“The total token usage throughout the trace is available in the token_usage field of the trace info object.”
↩︎ Context distraction: so much context the model stops thinking - 7.
“Some tables might include overlapping columns or concepts that make it difficult for Genie to know which data to use in a response.”
↩︎ Context confusion: irrelevant material gets used“Some tables might include overlapping columns or concepts that make it difficult for Genie to know which data to use in a response.”
↩︎ Exam trap 3“If possible, remove unnecessary or overlapping tables or columns.”
↩︎ Checkpoint - 8.
“If your application spans radically different sub-domains (finance, devops, marketing, etc.), a single agent can become unwieldy or overloaded with functionality requirements.”
↩︎ Context confusion: irrelevant material gets used
Also cited
“With only an input and final response, you cannot tell which intermediate decision caused a failure.”
↩︎ Key concept“With only an input and final response, you cannot tell which intermediate decision caused a failure.”
↩︎ Checkpoint