What you will be able to do
- Restate an output's purpose, audience and required items before judging its details
- Check an output's accuracy against source material and its completeness and relevance against the request
- Classify a reviewed output as ready to use, requiring revision, or requiring human correction
- Handle a found defect in order: pinpoint it, decide who fixes it, request a targeted fix, and recheck
Key concept
Fitness for purpose — An output is fit for purpose when it does the job the request set out to do, for the audience it was meant for. Polish and completeness alone don't make it fit. Every other check in an evaluation is measured against that purpose.
1.Start from the purpose, not the draft
Evaluating a Claude output means judging what came back before anyone acts on it. A draft can read fluently, cover every heading and still fail. Code that runs can still fail, and a checklist with every section filled in can still be the wrong checklist. A reviewer who starts by reading the draft line by line ends up judging it against itself. The useful question is whether it does what it was asked to do.
The first step therefore happens before you read a single detail. Go back to the original request and restate three things: what the output is for, who will read it, and which items it must contain. Take an HR assistant reviewing an onboarding checklist. They first write down that it is for new hires in their first week, that it must name the systems those hires need access to, and that it must list the policy acknowledgements they have to sign. Only then do they open the draft. Every later check has a yardstick because of this step.
Professional standards work the same way, provided they are written down. Suppose a team expects a particular register for customer letters, or a set structure for incident reports. Those expectations become usable criteria only once someone has stated them. A vague sense that something feels off can't be applied consistently, by you or by a colleague reviewing the same output. The planning point carries over too. Writing down what good enough looks like before you ask Claude for anything is what makes the later evaluation reliable.
Checkpoint 1 of 8· Check yourself
A marketing manager asked Claude for a customer-facing product FAQ, and the draft has just come back. Before acting on it, what should the manager do first?
The first step is to restate the purpose, audience and required items from the original request. That gives every later check something to be measured against. If you start with the draft's details, you end up judging it against itself. A full, fluent draft can still fail its purpose, and a vague sense that something is off can't be applied consistently.
“AI cannot serve a practice you have not described, and you cannot evaluate output against standards you have not made explicit.”Source: academy.claude.com
2.Check accuracy against the source material
Accuracy is checked against something outside the output: the handbook, the CRM report, the policy document or the spreadsheet the task was based on. Claude can produce text that is factually incorrect or inconsistent with the context it was given. A confident tone tells you nothing about whether a figure is right. The method is to compare each checkable detail (dates, names, system names, counts, policy titles) with its source, one at a time.
Some facts have no source in the conversation at all. Claude's knowledge comes from training and stops at a cutoff, and without tools it has nothing newer. A revised policy, this quarter's figures or a local regulation may be wrong simply because Claude never had them. For tasks like these, you have to bring the knowledge yourself, through context, documents or search. Where the source never reached Claude, the reviewer is the only one who can check the claim.
How much accuracy matters depends on the purpose you restated in the first step. Citation accuracy can be critical for a medical application and much less important for a casual chatbot. A patient-facing notice and an internal brainstorming list don't need the same scrutiny. Detecting fabricated figures and citations is a skill of its own, covered under hallucination detection and verification. In this lesson, accuracy is one dimension of the overall judgement.
Checkpoint 2 of 8· Check yourself
Claude has drafted a staff summary of the parental-leave policy. The policy was revised last month, and the revised document was never shared in the conversation. At the end, Claude writes: "I have double-checked all dates and entitlements." How should the reviewer establish whether the summary is accurate?
Accuracy is established by comparing the output against an outside source, one detail at a time. Claude never had the revised policy, so the reviewer has to bring that knowledge. Claude's statement that it double-checked, a repeat pass by Claude, and a confident tone are all part of the output, so none of them can confirm the figures.
“flag whether you can lean on the model's knowledge, or whether you need to bring the knowledge yourself via context, documents, or search.”Source: academy.claude.com
3.Check completeness, relevance and audience fit
Once the details are verified, compare the draft with the required items you listed in the first step. Two kinds of gap show up here. Something the request required may be missing. Or something may be present that the request never asked for, such as a section of general advice that pads the checklist and pushes the required items further down. Both count against fitness for purpose, because completeness and relevance are each measured against the request, not against how thorough the draft looks.
Audience fit is the last dimension. A useful test is whether the output uses language and patterns its reader would recognise. A summary written for engineers can be accurate and complete and still useless to a finance director. Most outputs need judging on several dimensions at once, and a draft that passes one can fail another.
| Dimension | Measured against | Typical failure |
|---|---|---|
| Accuracy | Source material: handbook, report, data file | A misstated count or outdated policy name |
| Completeness | Required items listed in the original request | A required step or section is missing |
| Relevance | The request's stated purpose | Off-topic content that dilutes the required items |
| Audience fit | Who will read it and what language they use | Correct content pitched at the wrong reader |
Checkpoint 3 of 8· Match them up
An HR reviewer finds four problems in a draft onboarding checklist for new hires. Match each evaluation dimension to the problem that fails it.
Tap a term, then the definition that fits it.
Each dimension is measured against something different. Accuracy is checked against the source material, completeness against the required items, relevance against the request's purpose, and audience fit against the reader. A draft that passes one dimension can still fail another, so each needs its own check.
“Most use cases need multidimensional evaluation along several success criteria.”Source: platform.claude.com
4.Classify the output: ready, revise, or human correction
Every evaluation ends with a decision about what happens to the output next. Ship it, fix it, or stop it. There are three classes, and they are separated by one question: who can close the gap?
| Class | What it means | Example |
|---|---|---|
| Ready to use | Meets every requirement with no defects; can go to its audience as it is | A checklist whose items, names and dates all match the request and the handbook |
| Requires revision | Has defects Claude can fix once given specific feedback or information you can supply | Wrong tone, a missing section, or a wrong figure when you hold the correct one |
| Requires human correction | Has a problem Claude cannot fix from what it has: facts no one supplied, or a judgement a person must make | A policy detail that exists only in an unreleased memo, or a call on which candidate to reject |
The line between revision and human correction is not how serious the error is. It is whether Claude can fix the problem with what it can be given. A wrong figure you can supply means revision. Here is the opposite case. A product owner has given feedback three times, and each version reintroduces a wrong warranty period because the warranty terms were never uploaded. Another round of feedback won't help. Either supply the source, or have a person correct the text. Decisions that call for human judgement fall in the human-correction class whatever Claude is given, because the person who signs off owns the result in full. Classification answers what happens to the draft. How hard to look before you classify depends on the stakes, which is the subject of review depth and escalation.
Checkpoint 4 of 8· Check yourself
A reviewer finds two problems in Claude's draft staff benefits notice. The pension contribution rate is badly wrong, but the reviewer has the payroll handbook that states the correct rate. Separately, a small eligibility date depends on a board decision recorded only in a memo that can't be shared yet. How should each problem be classified?
The deciding question is who can close the gap, not how serious the error is. The reviewer can supply the correct pension rate, so Claude can fix it, which makes it a revision. The eligibility date depends on a memo Claude can't be given, so more feedback won't fix it and a person has to correct it.
“flag whether you can lean on the model's knowledge, or whether you need to bring the knowledge yourself via context, documents, or search.”Source: academy.claude.com
Checkpoint 5 of 8· Exam question
A training coordinator reads a guide that sorts reviewed Claude outputs into three groups: ready to use, requires revision, and requires human correction. Which description matches "requires human correction"?
Correct answer: C — The output holds a problem Claude cannot fix, such as missing facts or a judgement that a person must make.
- A. Incorrect: that describes the ready-to-use category, where nothing needs changing before the output reaches its audience.
- B. Incorrect: that describes the revision category, where Claude can repair issues once feedback names them.
- C. Correct: this category covers defects that depend on information or judgement Claude lacks, so someone with that knowledge or accountability must correct it.
- D. Incorrect: length is a requirement to adjust through prompting. It does not need a person to take over.
Checkpoint 6 of 8· Exam question
An HR assistant receives a draft onboarding checklist from Claude. The team wants to judge the output for fitness for purpose and decide whether it can be used. Which sequence of evaluation steps should the assistant follow?
Correct answer: A — Restate purpose, audience and required items → Compare dates and names with the HR handbook → Check for missing or off-topic items → Classify draft as ready, revise or human fix
- A. Restating the purpose, audience and required items sets the standard that the fact check and the completeness check are measured against. The classification comes last because it is a conclusion drawn from all the findings.
- B. Comparing dates and names with the handbook before restating the purpose starts on details with no standard. The assistant does not yet know which items the checklist must contain or who it is for, so it may spend effort on details that do not matter for its purpose.
- C. Checking for missing or off-topic items before restating the purpose has nothing to judge omissions or drift against. The required items and the audience have not yet been set out, so the completeness check rests on guesswork.
- D. Comparing facts and then checking for omissions before restating the purpose means both checks run without a stated standard. The purpose is written down only after the findings exist, so it cannot shape them, and the findings may not reflect what the checklist is meant to do.
5.Handle a defect: name it, route it, fix it, recheck
Finding a defect is not the same as dealing with it. Telling Claude only that the summary has a wrong number sends it hunting and invites new errors. A critique you would give a colleague is not the same as an instruction Claude can act on. Feedback that works points at the exact problem and supplies what is needed to fix it.
A sales operations analyst whose quarterly summary misstates a customer count works through it in this order:
- Pinpoint the exact sentence where the count differs from the CRM report.
- Decide who fixes it. The analyst holds the correct figure, so Claude can fix it once given that figure.
- Send Claude a targeted request that includes the CRM figure and asks it to correct that line.
- Recheck the revised line and the sentences around it against the CRM report.
The final step is wider than the fix itself. When a corrected figure changes, a total, a percentage change or a comparison in the next sentence may now be wrong. A revision can also bring back an error an earlier round had removed. Asking Claude to verify its previous statements can catch and correct inconsistencies, but the reviewer's own check against the source is what closes the loop.
Who fixes it depends on what the defect is. A wrong figure you can supply goes back to Claude as a targeted request. A missing fact no one has provided, or a judgement call, goes to a person. If you route the problem before naming it, a fixable problem can end up with a person, or an unfixable one can go back to Claude for another round that won't help.
Checkpoint 7 of 8· Check yourself
Which follow-up message to Claude is most likely to fix the defect without introducing new errors? A sales operations analyst's quarterly summary says there are 1,240 active customers, but the CRM report shows 1,420.
Feedback that works points to the exact problem and gives Claude what it needs to fix it. Here that means the sentence and the correct CRM figure. Vague or general requests send Claude hunting and invite new errors. Claude doesn't have the CRM data, so regenerating from its own knowledge can't fix the count. After the fix, the analyst still rechecks the revised line and the sentences around it against the CRM report.
“Critique AI-generated designs using feedback specific enough to act on”Source: academy.claude.com
Checkpoint 8 of 8· Exam question
A sales operations analyst finds that Claude's draft quarterly summary misstates a customer count. The correct count is in a CRM report the analyst can open. Which sequence best handles this defect?
Correct answer: B — Locate the sentence that conflicts with the CRM → Decide Claude can fix it with the figure → Ask Claude to correct that line → Recheck the revised text against the CRM
- A. Deciding that Claude can fix the problem before locating the mismatched sentence means the decision rests on an undiagnosed defect. The analyst does not yet know whether it is one simple count error, so the choice of who fixes it is premature.
- B. The defect has to be located first, then the analyst decides who fixes it, then the targeted request goes to Claude with the CRM figure, and the revised text is checked against the same source at the end.
- C. Asking Claude to correct the line before deciding who should fix it means the request goes out without a judgment on whether Claude is the right fixer. The analyst should first confirm that a simple figure correction suits Claude, and only then send the request.
- D. Sending the correction request before locating the conflicting sentence leaves the request vague, with no specific line to point at. Claude may change unrelated parts of the draft, and the analyst has not yet judged whether Claude is the right one to fix it.
Sources
(8)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Code that runs can still fail.”
↩︎ Start from the purpose, not the draft“The default AI output is technically complete but often misses the point.”
↩︎ Key concept“fix it yourself, or describe it better?”
↩︎ Prediction - 2.
“Discernment means evaluating what AI gives back before acting on it.”
↩︎ Start from the purpose, not the draft“Diligence means owning the final result fully”
↩︎ Classify the output: ready, revise, or human correction - 3.https://academy.claude.com/courses/ai-fluency-for-creative-work/description-and-discernmentOfficial docs
“you cannot evaluate output against standards you have not made explicit.”
↩︎ Start from the purpose, not the draft“you cannot evaluate output against standards you have not made explicit.”
↩︎ Exam trap 1“AI cannot serve a practice you have not described, and you cannot evaluate output against standards you have not made explicit.”
↩︎ Checkpoint - 4.https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-hallucinationsOfficial docs
“Even the most advanced language models, such as Claude, can sometimes generate text that is factually incorrect or inconsistent with the given context.”
↩︎ Check accuracy against the source material“asking it to verify or expand on previous statements. This can catch and correct inconsistencies.”
↩︎ Handle a defect: name it, route it, fix it, recheck“asking it to verify or expand on previous statements. This can catch and correct inconsistencies.”
↩︎ Exam trap 4 - 5.
“Without tools, it has no access to any information after that date.”
↩︎ Check accuracy against the source material“bring the knowledge yourself via context, documents, or search.”
↩︎ Exam trap 2“flag whether you can lean on the model's knowledge, or whether you need to bring the knowledge yourself via context, documents, or search.”
↩︎ Checkpoint“flag whether you can lean on the model's knowledge, or whether you need to bring the knowledge yourself via context, documents, or search.”
↩︎ Checkpoint - 6.
“Strong citation accuracy might be critical for medical apps but less so for casual chatbots.”
↩︎ Check accuracy against the source material“How appropriate is its language for the target audience?”
↩︎ Check completeness, relevance and audience fit“Most use cases need multidimensional evaluation along several success criteria.”
↩︎ Exam trap 3“Most use cases need multidimensional evaluation along several success criteria.”
↩︎ Checkpoint - 7.https://academy.claude.com/courses/ai-fluency-for-builders/discernment-for-user-experienceOfficial docs
“Does the output use language and patterns a clinic admin would recognize?”
↩︎ Check completeness, relevance and audience fit“A good critique and an actionable AI description are different artifacts.”
↩︎ Handle a defect: name it, route it, fix it, recheck“Critique AI-generated designs using feedback specific enough to act on”
↩︎ Handle a defect: name it, route it, fix it, recheck - 8.
“Make the call: ship it, fix it, or stop it”
↩︎ Classify the output: ready, revise, or human correction