CertSafari
    Anthropic Claude Certified Associate – Foundations (CCAO-F) - December 7 onwards· Lessons

    Domain 2 · Lesson 4/21

    Evaluating Claude output for fitness for purpose

    Output Evaluation and Fitness for Purpose

    14 min read
    5% of exam
    8 sources
    Published 18 Nov 2026
    Docs as of 28 Oct 2026

    What you will be able to do

    • Restate an output's purpose, audience and required items before judging its details
    • Check an output's accuracy against source material and its completeness and relevance against the request
    • Classify a reviewed output as ready to use, requiring revision, or requiring human correction
    • Handle a found defect in order: pinpoint it, decide who fixes it, request a targeted fix, and recheck

    Key concept

    Fitness for purpose — An output is fit for purpose when it does the job the request set out to do, for the audience it was meant for. Polish and completeness alone don't make it fit. Every other check in an evaluation is measured against that purpose.

    1.Start from the purpose, not the draft

    Evaluating a Claude output means judging what came back before anyone acts on it. A draft can read fluently, cover every heading and still fail. Code that runs can still fail, and a checklist with every section filled in can still be the wrong checklist. A reviewer who starts by reading the draft line by line ends up judging it against itself. The useful question is whether it does what it was asked to do.

    The first step therefore happens before you read a single detail. Go back to the original request and restate three things: what the output is for, who will read it, and which items it must contain. Take an HR assistant reviewing an onboarding checklist. They first write down that it is for new hires in their first week, that it must name the systems those hires need access to, and that it must list the policy acknowledgements they have to sign. Only then do they open the draft. Every later check has a yardstick because of this step.

    Professional standards work the same way, provided they are written down. Suppose a team expects a particular register for customer letters, or a set structure for incident reports. Those expectations become usable criteria only once someone has stated them. A vague sense that something feels off can't be applied consistently, by you or by a colleague reviewing the same output. The planning point carries over too. Writing down what good enough looks like before you ask Claude for anything is what makes the later evaluation reliable.

    Checkpoint 1 of 8· Check yourself

    A marketing manager asked Claude for a customer-facing product FAQ, and the draft has just come back. Before acting on it, what should the manager do first?

    Sources123

    2.Check accuracy against the source material

    Accuracy is checked against something outside the output: the handbook, the CRM report, the policy document or the spreadsheet the task was based on. Claude can produce text that is factually incorrect or inconsistent with the context it was given. A confident tone tells you nothing about whether a figure is right. The method is to compare each checkable detail (dates, names, system names, counts, policy titles) with its source, one at a time.

    Some facts have no source in the conversation at all. Claude's knowledge comes from training and stops at a cutoff, and without tools it has nothing newer. A revised policy, this quarter's figures or a local regulation may be wrong simply because Claude never had them. For tasks like these, you have to bring the knowledge yourself, through context, documents or search. Where the source never reached Claude, the reviewer is the only one who can check the claim.

    How much accuracy matters depends on the purpose you restated in the first step. Citation accuracy can be critical for a medical application and much less important for a casual chatbot. A patient-facing notice and an internal brainstorming list don't need the same scrutiny. Detecting fabricated figures and citations is a skill of its own, covered under hallucination detection and verification. In this lesson, accuracy is one dimension of the overall judgement.

    Checkpoint 2 of 8· Check yourself

    Claude has drafted a staff summary of the parental-leave policy. The policy was revised last month, and the revised document was never shared in the conversation. At the end, Claude writes: "I have double-checked all dates and entitlements." How should the reviewer establish whether the summary is accurate?

    Sources456

    3.Check completeness, relevance and audience fit

    Once the details are verified, compare the draft with the required items you listed in the first step. Two kinds of gap show up here. Something the request required may be missing. Or something may be present that the request never asked for, such as a section of general advice that pads the checklist and pushes the required items further down. Both count against fitness for purpose, because completeness and relevance are each measured against the request, not against how thorough the draft looks.

    Audience fit is the last dimension. A useful test is whether the output uses language and patterns its reader would recognise. A summary written for engineers can be accurate and complete and still useless to a finance director. Most outputs need judging on several dimensions at once, and a draft that passes one can fail another.

    The dimensions of evaluation and what each is measured against
    DimensionMeasured againstTypical failure
    AccuracySource material: handbook, report, data fileA misstated count or outdated policy name
    CompletenessRequired items listed in the original requestA required step or section is missing
    RelevanceThe request's stated purposeOff-topic content that dilutes the required items
    Audience fitWho will read it and what language they useCorrect content pitched at the wrong reader

    Checkpoint 3 of 8· Match them up

    An HR reviewer finds four problems in a draft onboarding checklist for new hires. Match each evaluation dimension to the problem that fails it.

    Tap a term, then the definition that fits it.

    Sources67

    4.Classify the output: ready, revise, or human correction

    Every evaluation ends with a decision about what happens to the output next. Ship it, fix it, or stop it. There are three classes, and they are separated by one question: who can close the gap?

    The three classes and what separates them
    ClassWhat it meansExample
    Ready to useMeets every requirement with no defects; can go to its audience as it isA checklist whose items, names and dates all match the request and the handbook
    Requires revisionHas defects Claude can fix once given specific feedback or information you can supplyWrong tone, a missing section, or a wrong figure when you hold the correct one
    Requires human correctionHas a problem Claude cannot fix from what it has: facts no one supplied, or a judgement a person must makeA policy detail that exists only in an unreleased memo, or a call on which candidate to reject

    The line between revision and human correction is not how serious the error is. It is whether Claude can fix the problem with what it can be given. A wrong figure you can supply means revision. Here is the opposite case. A product owner has given feedback three times, and each version reintroduces a wrong warranty period because the warranty terms were never uploaded. Another round of feedback won't help. Either supply the source, or have a person correct the text. Decisions that call for human judgement fall in the human-correction class whatever Claude is given, because the person who signs off owns the result in full. Classification answers what happens to the draft. How hard to look before you classify depends on the stakes, which is the subject of review depth and escalation.

    Checkpoint 4 of 8· Check yourself

    A reviewer finds two problems in Claude's draft staff benefits notice. The pension contribution rate is badly wrong, but the reviewer has the payroll handbook that states the correct rate. Separately, a small eligibility date depends on a board decision recorded only in a memo that can't be shared yet. How should each problem be classified?

    Checkpoint 5 of 8· Exam question

    A training coordinator reads a guide that sorts reviewed Claude outputs into three groups: ready to use, requires revision, and requires human correction. Which description matches "requires human correction"?

    Checkpoint 6 of 8· Exam question

    An HR assistant receives a draft onboarding checklist from Claude. The team wants to judge the output for fitness for purpose and decide whether it can be used. Which sequence of evaluation steps should the assistant follow?

    Sources82

    5.Handle a defect: name it, route it, fix it, recheck

    Finding a defect is not the same as dealing with it. Telling Claude only that the summary has a wrong number sends it hunting and invites new errors. A critique you would give a colleague is not the same as an instruction Claude can act on. Feedback that works points at the exact problem and supplies what is needed to fix it.

    A sales operations analyst whose quarterly summary misstates a customer count works through it in this order:

    1. Pinpoint the exact sentence where the count differs from the CRM report.
    2. Decide who fixes it. The analyst holds the correct figure, so Claude can fix it once given that figure.
    3. Send Claude a targeted request that includes the CRM figure and asks it to correct that line.
    4. Recheck the revised line and the sentences around it against the CRM report.

    The final step is wider than the fix itself. When a corrected figure changes, a total, a percentage change or a comparison in the next sentence may now be wrong. A revision can also bring back an error an earlier round had removed. Asking Claude to verify its previous statements can catch and correct inconsistencies, but the reviewer's own check against the source is what closes the loop.

    Checkpoint 7 of 8· Check yourself

    Which follow-up message to Claude is most likely to fix the defect without introducing new errors? A sales operations analyst's quarterly summary says there are 1,240 active customers, but the CRM report shows 1,420.

    Checkpoint 8 of 8· Exam question

    A sales operations analyst finds that Claude's draft quarterly summary misstates a customer count. The correct count is in a CRM report the analyst can open. Which sequence best handles this defect?

    Sources74

    Sources

    (8)

    Every claim above is drawn from one of these pages, quoted as it was written on the date shown.

    1. 1.
      “Code that runs can still fail.”
      ↩︎ Start from the purpose, not the draft
      “The default AI output is technically complete but often misses the point.”
      ↩︎ Key concept
      “fix it yourself, or describe it better?”
      ↩︎ Prediction
    2. 2.
      “Discernment means evaluating what AI gives back before acting on it.”
      ↩︎ Start from the purpose, not the draft
      “Diligence means owning the final result fully”
      ↩︎ Classify the output: ready, revise, or human correction
    3. 3.
      “you cannot evaluate output against standards you have not made explicit.”
      ↩︎ Start from the purpose, not the draft
      “you cannot evaluate output against standards you have not made explicit.”
      ↩︎ Exam trap 1
      “AI cannot serve a practice you have not described, and you cannot evaluate output against standards you have not made explicit.”
      ↩︎ Checkpoint
    4. 4.
      “Even the most advanced language models, such as Claude, can sometimes generate text that is factually incorrect or inconsistent with the given context.”
      ↩︎ Check accuracy against the source material
      “asking it to verify or expand on previous statements. This can catch and correct inconsistencies.”
      ↩︎ Handle a defect: name it, route it, fix it, recheck
      “asking it to verify or expand on previous statements. This can catch and correct inconsistencies.”
      ↩︎ Exam trap 4
    5. 5.
      “Without tools, it has no access to any information after that date.”
      ↩︎ Check accuracy against the source material
      “bring the knowledge yourself via context, documents, or search.”
      ↩︎ Exam trap 2
      “flag whether you can lean on the model's knowledge, or whether you need to bring the knowledge yourself via context, documents, or search.”
      ↩︎ Checkpoint
      “flag whether you can lean on the model's knowledge, or whether you need to bring the knowledge yourself via context, documents, or search.”
      ↩︎ Checkpoint
    6. 6.
      “Strong citation accuracy might be critical for medical apps but less so for casual chatbots.”
      ↩︎ Check accuracy against the source material
      “How appropriate is its language for the target audience?”
      ↩︎ Check completeness, relevance and audience fit
      “Most use cases need multidimensional evaluation along several success criteria.”
      ↩︎ Exam trap 3
      “Most use cases need multidimensional evaluation along several success criteria.”
      ↩︎ Checkpoint
    7. 7.
      “Does the output use language and patterns a clinic admin would recognize?”
      ↩︎ Check completeness, relevance and audience fit
      “A good critique and an actionable AI description are different artifacts.”
      ↩︎ Handle a defect: name it, route it, fix it, recheck
      “Critique AI-generated designs using feedback specific enough to act on”
      ↩︎ Handle a defect: name it, route it, fix it, recheck

    Ready to test yourself?

    Practise Anthropic Claude Certified Associate – Foundations (CCAO-F) - December 7 onwards in quiz mode.

    Spotted a mistake, or was something unclear? Tell us.