When Extraction Retries Fail, and Fields That Help the Model Self-Correct

1.The limits of retry: information that isn't in the source

Retry with error feedback works when the correct answer is in the document and the model produced it badly. Examples are a value placed under the wrong object, a date in the wrong format, or a structural mix-up. The feedback points at the mistake, the model re-reads the source, and the fix is there to find.

It doesn't work when the information isn't in what the model was given. If a field's value appears only in an annual letter, a purchase order, or some other external document that was never supplied, no error message can help. Worse, repeatedly telling the model a required field is empty pushes it to fill the gap with something plausible. The hallucination guide gives the opposite advice: explicitly let Claude admit uncertainty, and instruct it to use only the provided documents rather than its general knowledge.

The Agent SDK's schema guidance puts the fix in the schema rather than the retry: if the task might not have all the information the schema needs, make those fields optional. An empty optional field is then a correct extraction, not a validation failure, and the pipeline can send the record to get the missing document or to a human. The content moderation cookbook has the same instinct. When a policy rule needs an external lookup of state gaming-license registries, the compiler marks it uncompilable instead of forcing out a rule it can't support.

Sources123

2.Self-correction fields: calculated_total, stated_total, conflict_detected

A note on sources: the field names in this section (calculated_total, stated_total, conflict_detected) come from the exam guide's description of this skill. The vendor documentation provided here doesn't name them. What the documentation does support is the principle behind them: make the model show its working so that errors can be seen.

The design works like this. Instead of extracting one total, the schema asks for two values. stated_total is the figure as printed on the document. calculated_total is the sum the model computes from the extracted line items. If they agree, the line items and the total back each other up. If they disagree, your validator has a specific, mechanical error to report, which is exactly the kind of feedback a retry can act on. This is the same idea as the hallucination guide's chain-of-thought verification: ask Claude to explain its reasoning step by step, because that can expose faulty logic. The outcome-grader cookbook makes the same point for graders: require concrete evidence, such as a traced formula, before passing anything.

The second field handles the case where the document disagrees with itself, for example an invoice whose printed total really doesn't match its printed line items. A conflict_detected boolean lets the model report that honestly instead of quietly picking one value. That matters for retry decisions. A mismatch with conflict_detected: false points to an extraction error worth retrying. A mismatch the model has flagged as a conflict in the source is the earlier page's case of a problem the document itself can't resolve, so it goes to review. The content moderation cookbook takes a similar approach to judgment calls: they go in clearly labelled fields (descriptions starting with JUDGMENT:), and null is reserved for cases that genuinely can't be decided.

Sources243

3.Feedback loops: detected_pattern and dismissed findings

Extraction quality isn't only about single records. Across many runs, you want to learn where the model is systematically wrong. The exam guide's example is a code-review pipeline whose structured findings include a detected_pattern field naming the code construct that triggered each finding. Again, the vendor documentation here doesn't define detected_pattern, so treat the name as the exam guide's.

The reasoning goes like this. When developers dismiss a finding, a free-text explanation tells you that one finding was wrong. A structured detected_pattern lets you group dismissals: if most findings triggered by one construct get dismissed, that construct is a candidate false-positive pattern, and you can fix the prompt's criteria for it rather than arguing case by case.

The documentation does show the surrounding practices. Claude Code's Code Review runs a verification step that checks results against actual code behavior to filter out false positives. Each finding is tagged with a severity level and comes with extended reasoning showing why Claude flagged it and how it verified it. On the prompt side, the outcome-grader cookbook warns that without a no-fire list, a grader thrashes on style nits, pre-existing issues, and scope creep. Dismissal data grouped by detected_pattern is how you decide, from evidence, what belongs on that list.

Sources54

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.If a required field keeps coming back empty, more retries with more forceful error feedback will eventually recover it.Why is that wrong?

   If the value isn't in the supplied source, feedback can't produce it and pressure invites fabrication. Make such fields optional and escalate or fetch the missing document.

   Covered in The limits of retry: information that isn't in the source

2. 2.When the source document contradicts itself, the model should quietly pick the most plausible value so the output passes validation.Why is that wrong?

   Letting the model report uncertainty, for example with a conflict_detected flag, reduces false information and tells the pipeline the problem is in the source, not the extraction.

   Covered in Self-correction fields: calculated_total, stated_total, conflict_detected