What you will be able to do
- Trace inconsistent or degraded output back to the scope, format or behavior requirement that the system prompt leaves unstated
- Set up a diagnosis-first workflow: compare failing and passing outputs and keep a regression test set before you edit any wording
- Choose revisions that constrain scope, format and behavior while leaving room for replies that should legitimately differ
- Spot conflicting and over-prescriptive rules, and fix them with priorities and templates instead of louder wording
Key concept
Underspecification — A system prompt is underspecified when it leaves open decisions that the application needs made one way. The model still has to make those decisions on every request, and it makes them differently each time, so the output comes out inconsistent or degraded.
1.How an underspecified system prompt shows up in output
Inconsistent output usually points to a missing requirement, not to a model that can't follow instructions. Take a system prompt that never says how long replies should be. The model still has to pick a length every time, so one reply runs to three paragraphs and the next is a bullet list. If the prompt never says what to do about policy questions, the model fills that gap too, and sometimes it guesses. Claude responds well to clear, explicit instructions. If you want a behavior the task doesn't strictly require, you have to ask for it. A vague prompt won't make the model infer it.
The quickest check is to imagine handing the prompt to a capable new employee who knows nothing about your norms. Picture a colleague with minimal context following it word for word. Every place where they would have to stop and ask is a place where the model will improvise, and it won't improvise the same way twice.
In a production system prompt, most of what goes unstated falls into three groups:
- Scope: which requests the assistant handles, and what falls outside its remit.
- Format: length, structure, and plain text versus markup.
- Behavior: what to do when the answer is uncertain, unknown, or belongs to someone else.
Underspecification has a quieter form too, called letter-over-spirit: the model obeys an instruction to the letter and misses its purpose, so the instruction was honored and the intent was not. "Make this shorter" applied to a draft whose real problem is its structure gives you a shorter draft that still doesn't work. The fix is to state the goal next to the instruction. Explaining why a constraint exists gives the model something to generalize from. A rule that only says "never use ellipses" covers ellipses and nothing else. If the prompt adds that the output will be read aloud by a text-to-speech engine, the model can apply the same reasoning to other symbols that a speech engine can't pronounce.
Checkpoint 1 of 8· Check yourself
A support assistant's whole system prompt is "You are a helpful support assistant for Acme." Some replies are three paragraphs and others are bullet lists. On refund-policy questions it sometimes makes up an answer. What is the best first diagnosis and fix?
Inconsistent output usually means a requirement is missing, not that the model lacks capability. A new colleague following this prompt would have to ask about length, format and policy questions, so the model improvises on each of these and does it differently each time. Stating the requirements closes those gaps. Emphasis and a lower length cap leave them open.
“Show your prompt to a colleague with minimal context on the task and ask them to follow it.”Source: platform.claude.com
2.Diagnose before you edit the wording
Editing the wording before you know what varies is guesswork. Start by collecting outputs. Pull failing outputs and passing outputs for similar inputs and put them side by side. Then ask which unstated requirement separates them. Suppose a ticket classifier formats its answer correctly for single-issue tickets but switches to a list when a ticket raises two problems. The prompt never said what to do with multi-issue tickets, and that missing requirement is what the revision has to state.
Prompt engineering assumes you already have success criteria and a way to test against them empirically, and that you have them before you start revising. In practice, every failing input you find becomes a test case. Add it to the existing set and rerun each revision against both the old cases and the new ones. That way a change that fixes this week's tickets but breaks last month's shows up as a regression and doesn't slip into production. Good evaluation sets deliberately include hard inputs and label them, such as an angry customer, a complex multi-step issue, or a compliment that reads like a complaint. One question to ask of every candidate revision is whether the format stays consistent across repeated attempts. Test each change on its own, so you know which edit actually helped.
Three tempting first moves skip the diagnosis:
- Moving up a model tier. Formatting consistency is something a prompt can control. Model choice is the lever for latency and cost; see model tier selection. A bigger model given the same vague prompt still has to guess.
- Adding a rule for every complaint and shipping at once. Longer, more complex prompts are not automatically better. The complaint tickets show that a problem exists, but they can't show that a new rule fixed it without breaking anything else.
- Regenerating the prompt wholesale from a metaprompt. The metaprompt recipe is a way to produce a first draft when you don't have one. Replacing a working prompt discards whatever it already got right, and reading ten fresh outputs can't tell you whether previously passing cases still pass.
Checkpoint 2 of 8· Check yourself
Users say a summarization assistant sometimes returns a table where they expected prose. Before you change any wording in its system prompt, what should you do first?
Comparing failing and passing outputs shows which missing requirement causes the variation. Rerunning the revision on old and new cases catches regressions. A bigger model still has to guess when the prompt is vague. Fixing only the complaint tickets, or reading ten fresh outputs, can't show that cases which passed before still pass.
“Pro tip: Start simple and add complexity only when needed. Test each addition to see if it actually improves results.”Source: claude.com
Checkpoint 3 of 8· Exam question
A team's classification assistant, called via the Messages API, produces inconsistent output formatting across tickets. Before any prompt edits are made, which two steps should come first? Choose two.(Select 2)
Correct answers: A, B — Gather failing and passing outputs for similar inputs and identify which unstated requirement differs between them before changing wording.; Add the failing inputs to the existing test set, so each revision is rerun against old and new cases and regressions show up.
- A. Correct: comparing failing and passing cases localises which requirement the prompt leaves unstated, so the revision targets the real gap instead of guessing.
- B. Correct: putting the failing inputs into the test set before editing gives a repeatable measure, so each revision can be checked for fixing the failures without breaking cases that already worked.
- C. Incorrect: upgrading the model tier first changes cost and latency without diagnosing the cause. Formatting inconsistency usually traces to missing instructions, which a larger model would still have to guess.
- D. Incorrect: stacking one rule per complaint with no evaluation risks conflicting instructions and no way to detect regressions. Complaints are symptoms, not a test of the revised prompt.
- E. Incorrect: wholesale regeneration discards the parts that work and reading ten samples is too small and informal to show whether behavior improved.
3.Constraining scope, format and behavior without over-constraining
A good revision states each missing decision as a positive default and leaves alone the variation that serves users. Take an internal IT helpdesk assistant whose whole system prompt is a single role line. The revision does three things. It names the topics the helpdesk covers, which constrains scope. It tells the assistant to say when it can't confirm something, such as a licensing entitlement, and to route that question to a person instead of guessing, which constrains behavior. And it sets a default of short plain-text replies, which constrains format. For narrow deployments, Anthropic's own guidance for Claude Opus 5 recommends constraining scope explicitly, along the lines of "deliver what was asked, at the scope intended". Giving explicit permission to acknowledge uncertainty, instead of speculating, also reduces made-up answers.
Phrasing matters as much as content. Telling Claude what to do works better than telling it what not to do: "write in short plain-text paragraphs" steers more reliably than "never use bullet points." Bans also over-reach. A rule that forbids discussing refunds at all blocks the legitimate answer that refunds are handled by a human agent, along with the speculation it was meant to stop. A richer persona fixes none of these problems either. Giving the assistant a name, a backstory and a friendly voice says nothing about which questions it should take or how long its replies should be. Being explicit about the perspective and the task often works better than assigning a role.
Over-constraint means forcing one fixed shape onto replies that ought to differ. Suppose a single template (greeting, three-sentence answer, closing) is applied to everything. A refusal gets padded and a multi-step troubleshooting reply gets truncated. When you use examples to show format, Anthropic advises making them diverse: they should cover edge cases and vary enough that the model doesn't copy accidental patterns. A default plus a stated reason for it allows a longer reply when the task really needs one. A rigid template does not.
| Dimension | Underspecified | Over-constrained | Well-constrained |
|---|---|---|---|
| Scope | "Answer user questions" | Refuse any topic not on a short fixed list, with no route onward | Lists covered topics and says what happens to the rest |
| Format | Nothing said; length and structure drift | One template for every reply type, including refusals | Short plain-text default, with the reason stated |
| Behavior on unknowns | Nothing said; the model speculates | Bans the whole subject | Says it can't confirm and hands off to a person |
| Role | Generic one-line role | Long persona with a backstory standing in for requirements | One-sentence role plus explicit task and perspective |
Checkpoint 4 of 8· Match them up
An IT helpdesk team is reviewing five candidate lines for its system prompt. Match each line to the effect it would have on replies.
Tap a term, then the definition that fits it.
A good revision states missing decisions as positive defaults, such as saying it can't confirm something and handing off to a person. A vague role line leaves those decisions open. A subject ban, a single template for every reply, and a persona each over-constrain in a different way, or stand in for the requirements without stating them.
“Tell Claude what to do instead of what not to do”Source: platform.claude.com
Checkpoint 5 of 8· Exam question
A retail company deploys a customer-support assistant through the Messages API with the system prompt: "You are a helpful assistant for Acme. Answer customer questions." Reviewing transcripts, you find replies that vary between long paragraphs and bullet lists, and some replies speculate about refund eligibility. Which revision best addresses the underspecification without over-constraining legitimate variation?
Correct answer: A — List the topics Acme support covers, hand refund and policy questions it cannot confirm to a human agent, and default to short replies in plain text.
- A. Correct: the transcripts show three gaps in scope, format and handling of uncertain cases. Listing covered topics, sending unconfirmable refund and policy questions to a person, and setting a default reply style closes each gap while leaving room for length to follow the question.
- B. Incorrect: banning bullet lists removes a format that is useful for step-by-step help, and barring refund discussion stops customers from getting policy answers Claude can confirm. This over-constrains legitimate variation, and it gives no escalation path for cases Claude cannot confirm.
- C. Incorrect: a detailed persona adds name, tone and backstory but sets no scope, no reply format and no rule for refund questions Claude cannot confirm. The mixed formats and the speculation about refund eligibility would continue.
- D. Incorrect: a single fixed template makes replies uniform but over-constrains them. Multi-step troubleshooting would be cut short, refusals would be padded to the same shape, and it still gives no rule for refund questions Claude cannot confirm.
4.Conflicting and over-prescriptive rules
When output swings between two extremes, look for two rules that pull against each other. Take a document-summary prompt that requires both full coverage of every section and brevity. The model can't satisfy both, so it settles the conflict differently on each run. Some runs produce a terse paragraph that drops something important. Others produce an exhaustive list. Neither run is a malfunction. Each is one valid reading of a contradictory prompt.
The fix is to decide which rule wins, and where, in the prompt itself. Two revisions work together here. The first is a priority order: name the items that must always be covered in full and allow everything else to be compressed to one line. The second is an output template with labelled sections, so coverage no longer depends on what the model decides to mention. Give the template a fixed line to write when an item is absent, so a missing section is reported explicitly and can't silently drop out. The same idea appears in a recommended extraction prompt that says to use null rather than guess when a metric isn't stated.
<competitor>
<name>Rival Inc</name>
<overview>A 50-word summary.</overview>
<swot>
<strengths>- Bullet points</strengths>
<weaknesses>- Bullet points</weaknesses>
<opportunities>- Bullet points</opportunities>
<threats>- Bullet points</threats>
</swot>
<strategy>A 30-word strategic response.</strategy>
</competitor>Three other fixes leave the conflict in place. Deleting the brevity and coverage rules gives up the constraints the application needs, so the output varies even more. Restating the same rules in bold at the end of the prompt only repeats the contradiction more loudly. Raising the output length budget doesn't help either. A one-paragraph summary that skips a key section didn't run out of room; it chose brevity, and nothing in the prompt said which rule should win.
Over-prescriptive rules cause the opposite failure: legitimate behavior gets suppressed. Anthropic tested a Claude Fable 5.2 approval rule that put "sending messages to people outside this conversation" among the actions needing sign-off. The model then held back about half of its routine requests to colleagues and other teams. Forceful wording has a similar effect. Recent models respond closely to the system prompt, so a "CRITICAL: You MUST" line written for older models can now overtrigger. The recommended fix is to soften the aggressive language.
A second rule added on top of the problem instruction doesn't reliably cancel it. Anthropic tested a prompt containing the sentence "You are the owner and long-term maintainer of this codebase, and its design is up to you." That ownership statement is what pushes the model to refactor: it consolidated duplicated code in about 75 percent of runs with the sentence, against under 10 percent without it. An earlier line in the same prompt, "No drive-by refactors, renames or cleanups", did not cancel the ownership statement, and the model still consolidated duplicated code in more than half of runs. The instruction causing the problem was the ownership sentence, not the scope rule. Where the user wants the code changed as little as possible, the fix is to leave the ownership sentence out instead of stacking a limit on top of it.
Checkpoint 6 of 8· Match them up
A document-summary prompt requires both full coverage of every section and brevity. Some runs give a terse paragraph that drops a key section, and others give an exhaustive list. Match each proposed revision to its effect.
Tap a term, then the definition that fits it.
The swings between extremes come from two rules that pull against each other. Only a priority order and a labelled template decide the conflict inside the prompt. Deleting the rules, adding emphasis, or allowing more length leaves the model to settle the conflict differently on each run.
“Precisely define the output format you want using JSON, XML, or custom templates so that Claude follows every output formatting element you require.”Source: platform.claude.com
Checkpoint 7 of 8· Check yourself
An agent's system prompt contains "CRITICAL: You MUST get approval before sending messages to people outside this conversation." The agent now holds back many routine requests to colleagues and other teams. Which revision best addresses this?
The rule is over-prescriptive and forcefully worded, so recent models apply it too widely. The fix is to reword the instruction causing the problem, naming the actions that really need sign-off and softening the language. A counter-rule stacked on top doesn't reliably cancel it, and repeating the rule or changing model tier leaves the broad wording in place.
“If you also write approval rules into your system prompt, be specific about the actions you want double-checked.”Source: platform.claude.com
Checkpoint 8 of 8· Exam question
A legal-tech team uses a system prompt for contract summaries that says: "Never include opinions. Always include every clause. Keep summaries brief." Outputs are sometimes a single paragraph that omits termination terms and sometimes a very long clause list. Which two revisions fix the diagnosed problem? Choose two.(Select 2)
Correct answers: A, B — Set a priority order, giving payment, termination and liability clauses full coverage and compressing the other clauses to one-line entries.; Define an output template with labelled sections, giving the model a fixed line to write when a clause type does not appear in the contract.
- A. Correct: the demand for complete clause coverage and the demand for brevity pull against each other, so the output swings between a thin paragraph and a long list. Ranking the goals, with full treatment for the key clauses and compression for the rest, resolves the conflict and keeps both goals.
- B. Correct: a labelled template with a fixed line for absent clause types gives every summary the same shape, so termination terms cannot silently drop out and the output no longer flips between a paragraph and a list.
- C. Incorrect: the brevity and no-opinion rules are legitimate requirements for contract summaries. Deleting them discards needs the team has, and leaving length and style to the model makes output even less predictable instead of specifying what to cover.
- D. Incorrect:
max_tokensonly caps output length. The short paragraphs that skip termination terms are not truncated output; they come from conflicting instructions, which a larger budget does nothing to resolve. - E. Incorrect: bold and repetition change emphasis, not content. The rules still demand both complete coverage and brevity, so the model has no basis for deciding which to favour and the inconsistent outputs continue.
Sources
(8)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/claude-prompting-best-practicesOfficial docs
“Claude responds well to clear, explicit instructions.”
↩︎ How an underspecified system prompt shows up in output“Your response will be read aloud by a text-to-speech engine, so never use ellipses”
↩︎ How an underspecified system prompt shows up in output“Tell Claude what to do instead of what not to do”
↩︎ Constraining scope, format and behavior without over-constraining“The fix is to soften any aggressive language.”
↩︎ Conflicting and over-prescriptive rules“If they’d be confused, Claude will be too.”
↩︎ Key concept“Tell Claude what to do instead of what not to do”
↩︎ Exam trap 2“Diverse: Cover edge cases and vary enough that Claude doesn’t pick up unintended patterns.”
↩︎ Exam trap 3“The fix is to soften any aggressive language.”
↩︎ Exam trap 4“Show your prompt to a colleague with minimal context on the task and ask them to follow it.”
↩︎ Checkpoint - 2.
“letter-over-spirit (the instruction was honored but the intent wasn't)”
↩︎ How an underspecified system prompt shows up in output“Repeating the instruction with more force won't close the gap.”
↩︎ Prediction - 3.
“Some ways to empirically test against those criteria”
↩︎ Diagnose before you edit the wording“Generate one with the metaprompt recipe from the Claude Cookbook.”
↩︎ Diagnose before you edit the wording“you can sometimes improve latency and cost more easily by choosing a different model”
↩︎ Exam trap 1 - 4.
“Edge case: Angry customer”
↩︎ Diagnose before you edit the wording - 5.https://claude.com/blog/best-practices-for-prompt-engineeringSecondary source
“Test each addition to see if it actually improves results.”
↩︎ Diagnose before you edit the wording“Don't over-engineer: Longer, more complex prompts are NOT always better.”
↩︎ Diagnose before you edit the wording“If the data is insufficient to draw conclusions, say so rather than speculating.”
↩︎ Constraining scope, format and behavior without over-constraining“Often, being explicit about what perspective you want is more effective”
↩︎ Constraining scope, format and behavior without over-constraining“If any metric is not clearly stated in the report, use null rather than guessing.”
↩︎ Conflicting and over-prescriptive rules“Pro tip: Start simple and add complexity only when needed. Test each addition to see if it actually improves results.”
↩︎ Checkpoint - 6.https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-opus-5Official docs
“Deliver what was asked, at the scope intended.”
↩︎ Constraining scope, format and behavior without over-constraining - 7.https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/increase-consistencyOfficial docs
“Precisely define the output format you want using JSON, XML, or custom templates”
↩︎ Conflicting and over-prescriptive rules“Precisely define the output format you want using JSON, XML, or custom templates so that Claude follows every output formatting element you require.”
↩︎ Checkpoint - 8.https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-fable-5-2Official docs
“led the model to hold back about half of its routine requests to colleagues and other teams.”
↩︎ Conflicting and over-prescriptive rules“A rule that limits scope doesn’t cancel it.”
↩︎ Conflicting and over-prescriptive rules“You are the owner and long-term maintainer of this codebase, and its design is up to you.”
↩︎ Conflicting and over-prescriptive rules“If you also write approval rules into your system prompt, be specific about the actions you want double-checked.”
↩︎ Checkpoint