What you will be able to do
- Pick the part of a business process that a release gate, prompt change or model change has to be evaluated on, and reject parts that are easy to score but don't measure the outcome
- Turn a business consequence into success criteria with a threshold for each dimension, set by how much each kind of error costs
- Keep an eval set matched to current production traffic as the product changes
- Combine code, model and human graders with production methods so the mix covers the deployed distribution
Key concept
Outcome-anchored evaluation — An evaluation is valid for a decision only if it scores the part of the process that determines the business outcome, against a known correct result taken from that process. Format, latency and how good the output looks are real dimensions, but on their own they don't show that the outcome happened.
1.Choosing the part of the process an eval must cover
Every Claude step inside a business process produces several things you could score: latency, token counts, whether the output parses, how clear an explanation reads. Almost all of them can be measured, and that makes it easy to gate a release on one of them. The real question is narrower. Which observable result tells you the business got what it needed? An eval is valid for a decision only if a better score on it means that result improved. So the order is fixed: define success first, then design the evaluation that measures performance against it.
What a system says and what a process produces are different things. A flight-booking agent can report that the flight is booked. The outcome is whether a reservation exists in the database. Grade the outcome, not the report. For a ticket-triage step, the outcome is whether the ticket reached the team that resolves it. For an invoice-extraction step, the outcome is whether the amount paid matches the invoice.
The process usually holds its own golden answers already, and these are the reference you compare model output against. A ticket that went straight to a team and was resolved there without reassignment has a confirmed correct queue. A reassigned ticket started with a disputed label. Replaying the confirmed cases through the new prompt and comparing each assignment with the team that resolved it tests exactly the outcome the business pays for. The other candidates measure something else. A JSON-and-valid-queue check confirms the output is well-formed, not that it is right. Latency and token cost are legitimate criteria, since response time and budget belong in most criteria sets, but they say nothing about whether routing improved. Expert ratings of the rationale grade a by-product nobody downstream uses.
The decision shapes how the suite is used, not what counts as success. For a prompt change or a model change, run the same task bank on both configurations and compare. Teams that already have evals can assess a new model and upgrade in days, as covered in model version migration. A release gate is a regression question: does the system still handle what it used to? Regression evals should pass at close to 100%, so a drop means something broke. Capability evals start at a low pass rate. They are a hill to climb, not a gate.
Checkpoint 1 of 7· Check yourself
A team is deciding whether to move a meeting-scheduling agent to a new model. At the end of each trial the agent tells the user something like "Your meeting is booked for Tuesday at 10:00." Which grader gives a valid basis for the migration decision?
The business outcome is the final state of the calendar, not what the agent says about it. Only checking the calendar system grades that outcome. A confirmation phrase, a polished message or a lower token count can all appear even when no event was created.
“the outcome is whether a reservation exists in the environment’s SQL database.”Source: www.anthropic.com
Checkpoint 2 of 7· Exam question
A support organization uses Claude through the Messages API to triage inbound tickets into one of 14 queues. A prompt rewrite is proposed, and the architect owns the release gate. The business outcome is that tickets reach the team that resolves them without reassignment. Which portion of the process should the gate evaluate?
Correct answer: D — Replay recent tickets that were never reassigned through the rewritten prompt and compare each assigned queue with the one that resolved it.
- A. Incorrect: latency and output tokens describe speed and cost. A faster, cheaper prompt can still send more tickets to the wrong team, so these figures say nothing about whether tickets reach the team that resolves them.
- B. Incorrect: agents can find a rationale clear and well written even when it supports the wrong queue. Rating the explanation does not measure whether the ticket lands with the resolving team, so it cannot gate the release.
- C. Incorrect: a response can parse cleanly and name a valid queue that is still the wrong one. Format compliance is a precondition for routing, not evidence that tickets reach the team that resolves them.
- D. Correct: the outcome is routing to the resolving team without reassignment, so the gate compares the prompt's assigned queue with the queue that resolved each ticket. Tickets that were never reassigned give a reliable ground-truth label.
2.Deriving the success criterion from the business consequence
A success criterion should be specific, measurable, achievable and relevant, and relevance is where the business process comes in. How much an error costs depends on where the output goes next. Citation accuracy can be critical in a medical app and minor in a casual chatbot. The same holds inside one workflow. When outputs feed different downstream actions, each action's cost of error sets its own criterion.
Most use cases need several success criteria at once. A strong criteria set lists the dimensions separately with a threshold for each: an F1 floor, a non-toxicity rate, a latency percentile, and a severity condition saying that 90% of errors should cause inconvenience rather than egregious harm. The severity condition matters most here. It admits that errors differ in cost and limits the expensive kind directly, instead of letting an average absorb them.
| Dimension | Threshold |
|---|---|
| Task quality | F1 score of at least 0.85 |
| Safety | 99.5% non-toxic outputs |
| Error severity | 90% of errors would cause inconvenience, not egregious error |
| Latency | 95% response time < 200ms |
Take invoice extraction ahead of payment scheduling. A wrong total pays the wrong amount. A wrong payee pays the wrong party. A wrong due date only shifts a reminder. The criterion therefore needs a separate threshold for each field, tightest on total and payee and loosest on due date. One combined accuracy figure lets correct due dates pull the average up while totals fail. The grader follows from the same reasoning. A payee or an amount has exactly one correct value, which is the clear-cut, categorical kind of answer that exact match handles well. A judge scoring "closeness" gives partial credit to a total with one wrong digit, even though that error is fully expensive. Mismatches on the costly fields can then be routed to human review checkpoints.
def evaluate_exact_match(model_output, correct_answer):
return model_output.strip().lower() == correct_answer.lower()Baselines tell you whether a target is achievable, not whether it is enough. Targets should rest on industry benchmarks, prior experiments, research or expert knowledge, and "5% better than the current baseline" is a reasonable achievability check. A release bar set at whatever the current prompt already scores turns today's failure rate into the definition of success. The required level comes from the cost of the error. The baseline only shows how far away that level is.
Checkpoint 3 of 7· Fill the gap
The payee field in an invoice-extraction step has exactly one correct value, and a near miss pays the wrong party. Which operator completes this grader so that only an output equal to the verified value passes?
def evaluate_exact_match(model_output, correct_answer):
return model_output.strip().lower() ? correct_answer.lower()Exact match passes only when the normalized output equals the verified value. That suits fields with one correct answer, where a partial match is still a full-cost error. in would pass a truncated or partial value. is compares object identity, not content. != reverses the check.
Checkpoint 4 of 7· Exam question
A finance team uses Claude to extract payee, invoice total and due date from supplier invoices before payments are scheduled. A wrong total triggers an incorrect payment, while a wrong due date only causes a minor reminder error. You must derive the success criterion for the evaluation from the business outcome. Choose two.(Select 2)
Correct answers: B, C — Grade the total and payee by exact match against human-verified values, and send mismatched or low-confidence extractions to manual review.; Set separate pass thresholds for each field, and make the bar tightest on the total and payee and loosest on the due date.
- A. Incorrect: closeness scoring gives high marks to amounts that are nearly right, and averaging the score across fields lets those near-misses pass. A total that is slightly off still causes an incorrect payment, so a verifiable amount needs an exact check rather than a graded judgement.
- B. Correct: payee and total have objectively verifiable values, so deterministic exact-match grading against human-verified labels is the appropriate grader. Sending mismatches and low-confidence extractions to review adds the control the business needs before payments are scheduled.
- C. Correct: the success criterion should mirror the asymmetric cost of errors. Fields that move money (total, payee) get the strictest bar, while the due date, whose errors only cause a minor reminder mistake, can have a looser one.
- D. Incorrect: a bar set from what the current prompt already achieves encodes the status quo, not the business requirement. It cannot show whether the error rate on totals is acceptable for payments, and it only ratchets relative to the prompt rather than to the outcome.
- E. Incorrect: a single pass mark across all three fields lets many correct due dates mask a few wrong totals. A release could pass while still triggering incorrect payments, because the criterion does not reflect that total errors cost far more than due-date errors.
Sources1
3.Keeping the eval matched to the deployed distribution
An eval predicts production quality only while its cases look like production traffic. Design evals that mirror the real-world task distribution, and keep the mix of questions and difficulty close to what users actually send. An eval that doesn't match real usage gives false confidence, and that is worse than having no eval, because it keeps passing.
Drift happens when the business changes and the eval set doesn't. Suppose an order assistant's eval set was written six months ago and every release passes at 97%. The company then launches a subscription product. Subscription conversations become a growing share of traffic, and the eval contains almost none of them. The score stays high because it measures last year's traffic. Production monitoring catches this kind of drift after launch, and the complaints are that signal.
The fix is to bring the eval set back in line with current traffic. Source realistic tasks from the failures you see. Sample recent production conversations, including the new flows, and add them with verified labels. Then reweight the combined set so each intent's share matches its current traffic share. Report the new intent as its own score, the same way a support suite reports refunds, cancellations and escalations separately, so its weakness can't hide inside a healthy overall number. Keep the original cases. Those intents still carry traffic, and work the system already handles well belongs in the regression suite. Raising the threshold or swapping the grader leaves the stale distribution in place, so a stricter gate still measures the wrong traffic. Finding what an existing suite misses is covered in evaluation coverage gaps.
tweets = [
{"text": "This movie was a total waste of time. 👎", "sentiment": "negative"},
{"text": "The new album is 🔥! Been on repeat all day.", "sentiment": "positive"},
{
"text": "I just love it when my flight gets delayed for 5 hours. #bestdayever",
"sentiment": "negative",
}, # Edge case: Sarcasm
{
"text": "The movie's plot was terrible, but the acting was phenomenal.",
"sentiment": "mixed",
}, # Edge case: Mixed sentiment
# ... 996 more tweets
]Edge cases belong in the distribution in proportion to how often they occur. Typical ones are irrelevant or missing input, overly long input, poor or harmful user input, and ambiguous cases where even humans would disagree. When privacy rules block testing on real inputs, Claude can generate synthetic test cases from the prompt template's variables. Those cases still have to resemble what production sends.
Checkpoint 5 of 7· Check yourself
Since a returns portal launched, every release of a ticket-triage step has still passed its eval at 96%, but complaints about misrouted returns tickets keep rising. Returns tickets now carry a growing share of traffic. What should the team change before trusting the next gate?
The score stays high because the set no longer matches current traffic. The fix is to add labelled production cases, weight each intent by its current traffic share, and keep the old cases as regression coverage. A stricter gate still measures the stale distribution, and equal weighting doesn't mirror real traffic either.
“try to have the distribution in your eval represent ~ the real life distribution of questions and question difficulties”Source: platform.claude.com
Checkpoint 6 of 7· Exam question
Six months ago a team built an eval set for its Claude order-assistant, and every release has passed it with 97 percent. Support now reports rising complaints after the company launched a subscription product, which accounts for a growing share of conversations. Which actions best restore the evaluation's link to the deployed distribution? Choose two.(Select 2)
Correct answers: A, C — Sample recent production conversations, including the subscription flows, and add them with verified labels to the eval set.; Reweight the combined set so each intent's share matches its current traffic share, and track subscription intents as their own slice.
- A. Correct: the set no longer covers a traffic segment that is growing, so adding labelled samples from live subscription conversations restores coverage of the deployed distribution.
- B. Incorrect: the stated problem is missing coverage rather than grader strictness. A new judge scoring the same cases leaves the subscription gap in place.
- C. Correct: aligning intent weights with current traffic and reporting subscription as its own slice makes the aggregate score reflect reality and exposes a weak segment that an average would hide.
- D. Incorrect: the original order cases still represent much of live traffic. Deleting them would remove regression protection for flows that remain important.
- E. Incorrect: a stricter threshold on the same cases still measures the old distribution. The gate can reach 99 percent while subscription flows remain untested.
4.Selecting the test-methodology mix
Choose a grader for each dimension of the criterion, and automate wherever the task allows. Code-based checks are fast, cheap and reproducible, but they break on valid variations of the expected answer. Model-based graders handle open-ended output but are non-deterministic and need calibration against human graders. Human grading is the gold standard but is slow and expensive. Grading cost comes back on every run, so a large eval graded automatically usually beats a small hand-graded one. Questions can often be restructured, for example as multiple choice, so that code can grade them. Rubric and bias design for model graders is covered in LLM-as-judge design.
| Grader | Typical methods | Use it for | Weakness |
|---|---|---|---|
| Code-based | String match, binary tests, outcome verification, tool call verification | Fields and outcomes with one correct answer | Brittle to valid variations |
| Model-based | Rubric-based scoring, pairwise comparison, reference-based evaluation | Tone, open-ended and freeform output | Non-deterministic; needs human calibration |
| Human | SME review, spot-check sampling, inter-annotator agreement | Calibrating model graders; subjective outputs | Expensive and slow |
No single offline suite can cover the whole deployed distribution, so the mix includes production methods too. Automated evals run in CI on every change and model upgrade. Production monitoring detects drift and real-world failures nobody anticipated. A/B tests validate significant changes against actual user outcomes once there is enough traffic. Weekly transcript reading and user-feedback triage catch what the other methods miss. Systematic human studies are reserved for calibrating model graders and for judging subjective outputs. Each layer, like a slice in the Swiss cheese model, catches failures that slipped through another.
Monitoring is reactive. Problems reach users before anyone sees the signal, and live traffic comes without ground-truth labels for grading. The automated suite is the first line of defense, and monitoring supplies the real distribution the suite is refreshed from.
Checkpoint 7 of 7· Match them up
No single offline suite covers the deployed distribution of a ticket-triage step. Match each layer of the methodology mix to the job it does.
Tap a term, then the definition that fits it.
Each layer catches what the others miss. Automated evals are the first line of defense on every change. Monitoring finds drift and unexpected failures after launch. A/B tests need real traffic. Human studies are slow and expensive, so they are kept for calibrating graders and judging subjective output.
“Production monitoring kicks in post-launch to detect distribution drift and unanticipated real-world failures. A/B testing validates significant changes once you have sufficient traffic.”Source: www.anthropic.com
Sources
(4)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Building a successful LLM-based application starts with clearly defining your success criteria and then designing evaluations to measure performance against them.”
↩︎ Choosing the part of the process an eval must cover“What is the acceptable response time for the model?”
↩︎ Choosing the part of the process an eval must cover“Strong citation accuracy might be critical for medical apps but less so for casual chatbots.”
↩︎ Deriving the success criterion from the business consequence“Most use cases need multidimensional evaluation along several success criteria.”
↩︎ Deriving the success criterion from the business consequence“90% of errors would cause inconvenience, not egregious error”
↩︎ Deriving the success criterion from the business consequence“It’s a simple, unambiguous metric that’s perfect for tasks with clear-cut, categorical answers”
↩︎ Deriving the success criterion from the business consequence“Base your targets on industry benchmarks, prior experiments, AI research, or expert knowledge.”
↩︎ Deriving the success criterion from the business consequence“Be task-specific: Design evals that mirror your real-world task distribution. Don’t forget to factor in edge cases!”
↩︎ Keeping the eval matched to the deployed distribution“More questions with slightly lower signal automated grading is better than fewer questions with high-quality human hand-graded evals.”
↩︎ Selecting the test-methodology mix“Automate when possible: Structure questions to allow for automated grading (for example, multiple-choice, string match, code-graded, LLM-graded).”
↩︎ Selecting the test-methodology mix“Most use cases need multidimensional evaluation along several success criteria.”
↩︎ Exam trap 2“Be task-specific: Design evals that mirror your real-world task distribution. Don’t forget to factor in edge cases!”
↩︎ Exam trap 3 - 2.
“the outcome is whether a reservation exists in the environment’s SQL database”
↩︎ Choosing the part of the process an eval must cover“Regression evals ask, “Does the agent still handle all the tasks it used to?” and should have a nearly 100% pass rate.”
↩︎ Choosing the part of the process an eval must cover“They should start at a low pass rate, targeting tasks the agent struggles with and giving teams a hill to climb.”
↩︎ Choosing the part of the process an eval must cover“Can create false confidence if it doesn’t match real usage patterns”
↩︎ Keeping the eval matched to the deployed distribution“Production monitoring kicks in post-launch to detect distribution drift and unanticipated real-world failures.”
↩︎ Keeping the eval matched to the deployed distribution“Source realistic tasks from the failures you see.”
↩︎ Keeping the eval matched to the deployed distribution“a customer support eval suite might test refunds, cancellations, and escalations.”
↩︎ Keeping the eval matched to the deployed distribution“Like the Swiss Cheese Model from safety engineering, no single evaluation layer catches every issue.”
↩︎ Selecting the test-methodology mix“Reactive; problems reach users before you know about themSignals can be noisyRequires investment in instrumentationLacks ground truth for grading”
↩︎ Selecting the test-methodology mix“the outcome is whether a reservation exists in the environment’s SQL database”
↩︎ Key concept“the outcome is whether a reservation exists in the environment’s SQL database”
↩︎ Exam trap 1“capability evals with high pass rates can “graduate” to become a regression suite that is run continuously to catch any drift”
↩︎ Exam trap 4“the outcome is whether a reservation exists in the environment’s SQL database.”
↩︎ Checkpoint“Production monitoring kicks in post-launch to detect distribution drift and unanticipated real-world failures. A/B testing validates significant changes once you have sufficient traffic.”
↩︎ Checkpoint - 3.https://platform.claude.com/cookbook/misc-building-evalsSecondary source
“A "golden answer" to which we compare the model output.”
↩︎ Choosing the part of the process an eval must cover“try to have the distribution in your eval represent ~ the real life distribution of questions and question difficulties.”
↩︎ Keeping the eval matched to the deployed distribution“Grading on the other hand is a cost you will incur every time you re-run your eval, in perpetuity”
↩︎ Selecting the test-methodology mix“try to have the distribution in your eval represent ~ the real life distribution of questions and question difficulties”
↩︎ Checkpoint - 4.https://platform.claude.com/cookbook/misc-generate-test-casesSecondary source
“maybe you aren't allowed to test on the ones you do have for privacy reasons. No worries -- Claude can make them up!”
↩︎ Keeping the eval matched to the deployed distribution