What you will be able to do
- Judge an output by whether it addresses the original request, not by how thorough it sounds
- Turn a request into an explicit checklist of what a correct answer must and must not contain
- Use specific, measurable criteria so different reviewers reach the same verdict
- Feed an evaluation back into a sharper follow-up request
1.Completeness is measured against the request
Accuracy asks whether what the response says is true. Completeness asks whether it does what was asked. The two fail separately. A response can be correct in every sentence and still answer a neighbouring question, cover two of the three things you asked for, or skip the constraint that mattered most. You only catch these failures by going back to the original request and checking the output against it. Reading the output by itself and asking whether it sounds right will not catch them.
The AI Fluency for Builders course warns that the default AI output is technically complete but often misses the point. It asks reviewers to judge work through several lenses, including correctness and fit, and not to stop at "it works". Anthropic's evaluation guidance uses the same test: how well does the response directly address the user's question or instruction?
Gaps can also be about perspective, not just missing items. The nonprofit course's research exercise asks reviewers to note any missing perspectives relevant to the communities they serve, and to flag information that is outdated or too general. The K-12 course frames Discernment as checking materials for accuracy, reading level and alignment to the learning objectives. Alignment to the goal behind the request counts as much as factual correctness.
Checkpoint 1 of 5· Check yourself
A project lead asks Claude for three things about a system migration: the main risks, a cost estimate and a timeline. The response has an excellent, detailed risk analysis and a clear timeline. How should the lead judge it?
Completeness is checked against what was asked for. A missing component is a gap no matter how strong the rest is, and noticing gaps is part of Discernment.
“Flag specific claims for verification, notice gaps, and question recency of information”Source: academy.claude.com
2.Turn the request into a checklist before you read
Evaluation practice offers a simple tool for this: write down what a correct answer looks like before you judge the output. Anthropic's eval cookbook calls this a "golden answer". It can be an exact expected value, or a description of a perfect answer that gives the reviewer something to score against.
The cookbook's examples show how specific such a description can be. One request asks for a workout with at least 50 reps of pulling leg exercises, at least 50 reps of pulling arm exercises and ten minutes of core. The golden answer spells out each component. It also says what disqualifies an answer: deadlifts count as pulling exercises, squats do not, and no other meaningful exercises are allowed. A checklist like this covers what must not appear as well as what must.
A second example shows that completeness does not always mean doing everything literally. Asked to send an email, an assistant with no ability to send email should decline to send it. Offering a draft is fine, but trying to send it, or asking for the recipient's address, is wrong. Here the complete answer handles the request correctly, which means not pretending to carry it out.
Anthropic's guidance on success criteria starts from the same point: be specific about what you want to achieve. If you gave Claude documents or other context, add a line to the checklist asking how effectively the response used that context. Ignoring material you supplied is a completeness failure in its own right.
Checkpoint 2 of 5· Check yourself
A user asks an assistant with no email capability to "send Jane an email asking her to meet at 9am". Which response matches the cookbook's golden answer?
The golden answer requires declining to send, because the assistant has no email capability, and explicitly allows a draft. Asking for the address or trying to send both fail.
“A correct answer should decline to send the email since the assistant has no capabilities to send emails.”Source: platform.claude.com
3.Shared criteria make reviewers agree
When several people review the same Claude draft and reach different verdicts, the usual cause is vague criteria, not unusual drafts. "Is this good?" invites everyone to apply their own private standard. Anthropic's guidance on success criteria says to make them specific and measurable. Instead of "good performance", name the actual target. Where a judgement is qualitative, use a well-defined scale, such as a 1–5 rating for coherence or an expert rubric with stated criteria. Purely qualitative impressions are hard to apply quickly or consistently.
One criterion is rarely enough. The same guidance says most use cases need evaluation along several success criteria at once, and that the right ones depend on the purpose. Citation accuracy might be critical for a medical app and matter much less for a casual chatbot. The questions below come from Anthropic's list of criteria worth considering. Agreeing in advance on which ones apply to a given output is what lets two reviewers reach the same verdict.
| Dimension | Question to ask of the output |
|---|---|
| Relevance | How well does the model directly address the user's questions or instructions? |
| Context use | How effectively does the model use provided context? |
| Tone and style | How appropriate is its language for the target audience? |
| Consistency | If a user asks the same question twice, how important is it that they get semantically similar answers? |
| Privacy | Can it follow instructions not to use or share certain details? |
Checkpoint 3 of 5· Match them up
Match each vague review comment to the specific criterion that would replace it
Tap a term, then the definition that fits it.
Each vague complaint maps to a named, checkable dimension. Turning impressions into stated criteria is what makes reviewers' verdicts consistent.
“Most use cases need multidimensional evaluation along several success criteria.”Source: platform.claude.com
Checkpoint 4 of 5· Exam question
An operations analyst asked six specific questions about a supplier's performance and received one flowing summary in reply. She is about to forward it to her manager. What should she do first?
Correct answer: B — Go back to her six questions and confirm that each one is actually answered somewhere in the summary.
- A. Reading for oddities catches things she already suspects. A question that was never answered at all leaves no trace in the text, so it is exactly what this kind of reading misses.
- B. Correct. Completeness is judged against what was asked, not against how the answer reads. A fluent summary can cover four of six questions and still feel finished, so checking question by question is what exposes the gap.
- C. Shortening compresses a summary whose coverage has not yet been established, and it can remove the very answers she has not verified are present.
- D. Her manager does not have her list of six questions, so he cannot tell what is missing, and the check she owns is passed to someone less able to make it.
Sources5
4.Feeding the evaluation back: the Description–Discernment loop
An evaluation should not end with a verdict. It should lead to a better next request. The AI Fluency courses describe a tight loop: you describe what you want, the model responds, you discern what is useful, and you describe again, more precisely this time. The gaps you found against your checklist become the content of the next prompt.
What matters is how specific the follow-up is. The K-12 exercise asks for one concrete revision prompt rather than a request to "make it better". Its example is to shorten the steps in a set of directions and add a picture cue for each one. Then compare the revision with the original. A follow-up that names the missing component or the failed criterion turns the evaluation into a fix. The Builders course asks a related question: when an output is not good enough, do you fix it yourself or describe it better? Both are valid, but only describing it better improves what Claude produces next time.
Checkpoint 5 of 5· Put it in order
Put the steps of the Description–Discernment loop in order
- 1.Describe what you want
- 2.Discern what is useful in the response
- 3.Describe again, more precisely this time
- 4.The model responds
The courses describe fluent AI use as this loop. Evaluating the response comes between the model's answer and your sharper follow-up.
“you describe what you want, the model responds, you discern what's useful, and you describe again, sharper this time.”Source: academy.claude.com
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.A long, accurate and thorough response is a complete one.Why is that wrong?
Completeness is judged against the original request. An accurate explainer that does not address the question asked, or leaves out a requested part, is incomplete however long it is.
Covered in Completeness is measured against the request
2.When reviewers disagree about a Claude draft, the fix is for a more senior person to make the call on overall quality.Why is that wrong?
Inconsistent verdicts usually come from vague criteria. Specific, measurable criteria and well-defined scales give every reviewer the same standard to apply.
Covered in Shared criteria make reviewers agree
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Evaluate AI-generated work across five lenses: correctness, quality, fit, experience, and responsibility”
↩︎ Completeness is measured against the request“When AI produces something that isn't good enough, what's your instinct: fix it yourself, or describe it better?”
↩︎ Feeding the evaluation back: the Description–Discernment loop“Code that runs can still fail. The default AI output is technically complete but often misses the point.”
↩︎ Exam trap 1 - 2.
“Note any missing perspectives relevant to your communities”
↩︎ Completeness is measured against the request“Flag specific claims for verification, notice gaps, and question recency of information”
↩︎ Checkpoint - 3.
“Use Discernment skills to evaluate AI-generated materials for accuracy, reading level, and alignment to your learning objectives”
↩︎ Completeness is measured against the request“Write one concrete revision prompt”
↩︎ Feeding the evaluation back: the Description–Discernment loop“you describe what you want, the model responds, you discern what's useful, and you describe again, sharper this time.”
↩︎ Checkpoint - 4.https://platform.claude.com/cookbook/misc-building-evalsSecondary source
“an example of a perfect answer meant to give a grader a point of comparison to base their scoring on”
↩︎ Turn the request into a checklist before you read“(such as deadlifts, but not such as squats which are a pushing exercise)”
↩︎ Turn the request into a checklist before you read“A correct answer should decline to send the email since the assistant has no capabilities to send emails.”
↩︎ Checkpoint - 5.
“Specific: Clearly define what you want to achieve.”
↩︎ Turn the request into a checklist before you read“How effectively does the model use provided context?”
↩︎ Turn the request into a checklist before you read“Purely qualitative evaluations are hard to assess quickly and at scale.”
↩︎ Shared criteria make reviewers agree“Strong citation accuracy might be critical for medical apps but less so for casual chatbots.”
↩︎ Shared criteria make reviewers agree“Measurable: Use quantitative metrics or well-defined qualitative scales.”
↩︎ Exam trap 2“How well does the model directly address the user's questions or instructions?”
↩︎ Prediction“Most use cases need multidimensional evaluation along several success criteria.”
↩︎ Checkpoint