What you will be able to do
- Rewrite a vague review instruction as an explicit, categorical criterion Claude can apply consistently
- Explain why qualitative filters such as "be conservative" or "only report high-confidence findings" are an unreliable way to control what a review reports
- Write report-versus-skip lists that define which issue categories a review surfaces and which it ignores
- Write review criteria that report bugs and security issues and skip minor style, instead of relying on confidence-based filtering
Key concept
Categorical review criteria — A review prompt should state the exact categories of issue that count as a finding, and the exact categories to leave out. Adjectives like "important" or "high-confidence" leave the model to choose its own line, so they are not criteria.
1.From vague instructions to explicit criteria
"Check that comments are accurate" sounds like a clear instruction, but it doesn't say what makes a comment inaccurate. Does a slightly stale wording count? A missing parameter description? A typo? Each reviewer, human or model, fills that gap with its own judgement, so the findings vary from run to run and many of them are things nobody wanted flagged. Compare it with an explicit criterion: "flag a comment only when the behaviour it claims contradicts what the code actually does." Now one test settles each case, and anything that fails the test is out of scope.
That comment rule is an illustration from the exam guide, but the principle comes straight from Anthropic's prompting guidance. The general advice is to be specific about the output you want and its constraints. When order or completeness matters, give instructions as numbered steps or bullets. Explaining *why* a rule exists also helps, because the model can apply the reasoning to cases you didn't list. Anthropic's blog describes specificity as structuring instructions around explicit guidelines and requirements, including clear constraints and the output structure you want.
In a code-review prompt, specificity means naming categories. The security-reviewer subagent in the Claude Code best-practices guide doesn't say "look for security problems". It lists the classes of vulnerability to look for and the form each finding should take.
You are a senior security engineer. Review code for:
- Injection vulnerabilities (SQL, XSS, command injection)
- Authentication and authorization flaws
- Secrets or credentials in code
- Insecure data handling
Provide specific line references and suggested fixes.Literalism makes explicit criteria matter even more on newer models. Anthropic's Sonnet 5 guide says the model reads prompts literally and won't infer requests you didn't make. If a category isn't stated, it doesn't get checked. If a vague instruction can be read two ways, you can't predict which reading you'll get.
Checkpoint 1 of 5· Check yourself
A team's review prompt says "check that comments are accurate" and the bot flags dozens of harmless wording quibbles. Which rewrite applies the guidance in this section?
Only the second option gives a condition that each comment either meets or doesn't. The other three keep the vague goal and add effort or confidence wording, which still leaves the model to decide where the line is.
“Be specific about the desired output format and constraints.”Source: platform.claude.com
2.Why "be conservative" is the wrong lever
It's tempting to deal with a noisy reviewer by turning down its eagerness with "be conservative", "only report high-confidence findings" or "don't nitpick". The Opus 4.8 and Sonnet 5 prompting guides describe what that does. The model treats the phrase as a filter and drops findings that it judges fall below the bar. The guides say precision typically rises while measured recall falls, even though the model's bug-finding ability has improved.
The real problem is that the bar is invisible. "High-severity" and "conservative" don't name a category, so you can't tell which real bugs were dropped, and you can't fix the prompt so that one kind of issue gets through while another doesn't. A confidence filter cuts across every category at once. A categorical criterion lets you choose category by category.
The guides give two remedies. First, if the model has to filter in a single pass, state the bar as concrete categories. Second, take filtering out of the finding step altogether: tell the model its job is coverage, have it attach a confidence and severity to each finding, and let a separate verification or ranking stage decide what gets reported.
| Instruction style | Example wording | Effect described in the model guides |
|---|---|---|
| Qualitative bar | "only report high-severity issues," "be conservative," "don't nitpick" | Followed faithfully: same investigation, fewer findings reported; recall can fall |
| Concrete single-pass bar | "report any bugs that could cause incorrect behavior, a test failure, or a misleading result; only omit nits like pure style or naming preferences" | The line is stated in categories, so you can check and adjust what gets left out |
| Coverage at the finding stage | "Report every issue you find ... include your confidence level and an estimated severity" | Filtering moves to a separate verification, deduplication or ranking stage |
Checkpoint 2 of 5· Check yourself
Your single-pass review bot must filter its own findings, with no downstream verifier. Which instruction follows Anthropic's guidance?
For self-filtering in a single pass, the guides recommend a concrete, categorical bar. The coverage wording in the last option assumes a filtering stage that this bot doesn't have, and the first two are the confidence and qualitative wording the guides warn about.
“report any bugs that could cause incorrect behavior, a test failure, or a misleading result; only omit nits like pure style or naming preferences.”Source: platform.claude.com
Sources4
3.Writing report-versus-skip criteria
Once the bar is stated in categories, you can write it down as two lists: what to always report, and what to never report. Claude Code's managed Code Review product is a worked example. By default it focuses on correctness, meaning bugs that would break production, and leaves out formatting preferences and missing test coverage. Teams change that default with a REVIEW.md file at the repository root. Its rules are added on top of the built-in correctness checks and tell the review what to flag, at what severity, and what to skip.
The example REVIEW.md in the docs doesn't use confidence wording anywhere. It lists categories to skip because another tool already handles them or because they aren't production code. It lists checks that must run on every PR because they guard against data leaks and tenant isolation failures.
## Do not report
- Anything CI already enforces: lint, formatting, type errors
- Generated files under `src/gen/` and any `*.lock` file
- Test-only code that intentionally violates production rules
## Always check
- New API routes have an integration test
- Log lines don't include email addresses, user IDs, or request bodies
- Database queries are scoped to the caller's tenantThe skip list works because each entry says *why* it's safe to skip. CI already enforces it, the code is generated, or the code is test-only. The same pattern covers local conventions. If a file mixes legacy naming styles, "flag naming that doesn't match the file" will produce noise. A criterion that names the project's canonical standard, or puts pure naming preferences in the skip list, removes that noise without hiding real defects.
Put together, a report-versus-skip criterion has three parts, and none of them is a confidence threshold.
1. Report: defects that change behaviour or expose data. Name them as categories. The docs' example reserves the top severity for findings "that would break behavior, leak data, or block a rollback", such as incorrect logic, unscoped database queries, PII in logs, and migrations that aren't backward compatible. For security, list the vulnerability classes, as the security-reviewer subagent does: injection, authentication and authorization flaws, secrets in code, insecure data handling. 2. Skip: minor style and local patterns. The same example says style, naming, and refactoring suggestions are Nit at most, and the single-pass wording in the model guides omits "nits like pure style or naming preferences". Add what is already enforced elsewhere (lint, formatting, generated files) and any local convention your team accepts, so the reviewer doesn't re-litigate it. 3. Say what happens to what you skip. Either drop it explicitly, as in a "Do not report" list, or demote it to a low severity or a capped number of nits, so a real bug is never silently discarded along with the noise.
This is what "explicit criteria" means in practice. If the reviewer reports a style quibble, you can point to the skip list entry that should have excluded it and fix that line. If it reports nothing for a real bug, you can see which report category was missing. With "only report high-confidence findings" you can do neither.
Checkpoint 3 of 5· Exam question
A team already tried adding "only report issues you are confident about" to a noisy category and saw no improvement in precision. An architect now wants to redesign the category's scope entirely rather than continuing to tune confidence language. Which redesign reflects the correct lesson from the earlier failed attempt?
Correct answer: B — Replace the confidence instruction with a list of the specific issue types that qualify for this category, and explicitly state which related issue types should be skipped.
- A. Incorrect. Message placement is not the reason a vague confidence instruction fails; the lesson from the earlier attempt is that the content of the instruction, not its location, needs to change.
- B. Correct. This replaces the confidence-based filter with categorical criteria defining exactly which issue types are in scope and which are excluded, addressing the root cause of the earlier failure rather than adjusting the phrasing of the same ineffective approach.
- C. Incorrect. Adding emphasis to the same confidence-based instruction does not change its fundamental nature; it still fails to define what makes an issue reportable, so the same failure mode would likely recur.
- D. Incorrect. A numeric threshold still asks the model to introspect on its own confidence rather than apply a defined categorical rule, and confidence self-reports of this kind have already been shown not to move precision in this scenario.
Checkpoint 4 of 5· Exam question
An architect is structuring a long review prompt that defines separate criteria for security, correctness, and style categories, each with its own inclusion rules and severity examples. Which structuring approach best helps Claude apply the right criteria to the right category without cross-contamination?
Correct answer: D — Wrap each category's criteria and examples in its own uniquely named XML tag, such as <security_criteria> and <correctness_criteria>, so the boundaries between categories are unambiguous.
- A. Incorrect. A single continuous paragraph without structural markers is more prone to the model blending or misapplying criteria across categories compared to explicitly tagged sections.
- B. Incorrect. Relying on bullet order alone to imply category membership is fragile and ambiguous; there is no explicit marker tying a given bullet to a specific category.
- C. Incorrect. Duplicating every category's full criteria in every section adds unnecessary length and redundancy without addressing the actual boundary-clarity problem, which structured tags solve directly.
- D. Correct. Wrapping each category's rules and examples in its own descriptively named XML tag gives the model clear, unambiguous boundaries between categories, which is the recommended way to structure prompts that mix multiple sets of instructions.
Checkpoint 5 of 5· Match them up
Match each REVIEW.md section from the docs example to what it does
Tap a term, then the definition that fits it.
The docs describe REVIEW.md as the place to say what gets flagged, at what severity, and how findings are reported. Each section of the example covers one of those jobs.
“Use it to say what your team wants flagged, at what severity, and how findings are reported.”Source: code.claude.com
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.Adding "be conservative" or "only report high-confidence findings" is a safe way to cut false positives without losing real bugs.Why is that wrong?
Newer models follow this wording faithfully. They still find the bugs but drop the ones below a bar you never defined, so recall falls without anyone noticing. Use a concrete, categorical bar instead.
Covered in Why "be conservative" is the wrong lever
2.When a harness has a separate verification stage, the finding-stage prompt should still ask the model to filter by importance and confidence.Why is that wrong?
The guides recommend telling the finding stage that its job is coverage. It should attach confidence and severity to each finding and leave the filtering to the downstream stage.
Covered in Why "be conservative" is the wrong lever
3.To keep a review focused on bugs and security, tell it to skip anything it is not sure is serious.Why is that wrong?
That is a confidence filter again. State the categories instead: report behaviour-breaking and data-leaking defects, and put style and naming in the skip list or at Nit severity.
Covered in Writing report-versus-skip criteria
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practicesOfficial docs
“Be specific about the desired output format and constraints.”
↩︎ From vague instructions to explicit criteria“Claude is smart enough to generalize from the explanation.”
↩︎ From vague instructions to explicit criteria - 2.https://claude.com/blog/best-practices-for-prompt-engineeringSecondary source
“Specificity in prompt engineering means structuring your instructions with explicit guidelines and requirements.”
↩︎ From vague instructions to explicit criteria - 3.https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-sonnet-5Official docs
“Claude Sonnet 5 interprets prompts literally and explicitly, particularly at lower effort levels.”
↩︎ From vague instructions to explicit criteria“Do not filter for importance or confidence at this stage - a separate verification step will do that.”
↩︎ Exam trap 2“report any bugs that could cause incorrect behavior, a test failure, or a misleading result; only omit nits like pure style or naming preferences.”
↩︎ Checkpoint - 4.https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-opus-4-8Official docs
“it may investigate the code just as thoroughly, identify the bugs, and then not report findings it judges to be below your stated bar.”
↩︎ Why "be conservative" is the wrong lever“moving confidence filtering out of the finding step often helps”
↩︎ Why "be conservative" is the wrong lever“only omit nits like pure style or naming preferences”
↩︎ Writing report-versus-skip criteria“be concrete about where the bar is rather than using qualitative terms like "important"”
↩︎ Key concept“Precision typically rises, but measured recall can fall even though the model's underlying bug-finding ability has improved.”
↩︎ Exam trap 1 - 5.
“By default, Code Review focuses on correctness: bugs that would break production, not formatting preferences or missing test coverage.”
↩︎ Writing report-versus-skip criteria“Both are additive on top of the default correctness checks.”
↩︎ Writing report-versus-skip criteria - 6.https://code.claude.com/docs/en/code-reviewOfficial docs
“Anything CI already enforces: lint, formatting, type errors”
↩︎ Writing report-versus-skip criteria“Style, naming, and refactoring suggestions are Nit at most.”
↩︎ Writing report-versus-skip criteria“Reserve Important for findings that would break behavior, leak data,”
↩︎ Writing report-versus-skip criteria“Style, naming, and refactoring suggestions are Nit at most.”
↩︎ Exam trap 3“Use it to say what your team wants flagged, at what severity, and how findings are reported.”
↩︎ Checkpoint - 7.https://code.claude.com/docs/en/best-practicesOfficial docs
“Injection vulnerabilities (SQL, XSS, command injection)”
↩︎ Writing report-versus-skip criteria