What you will be able to do
- Run prompt refinement as small, testable changes instead of large rewrites
- Write specific, measurable criteria for what a better answer looks like
- Check a refined prompt against varied, realistic inputs before others rely on it
- Choose where a proven prompt should live so it applies where it is needed
1.Change one thing, then test it
Prompt refinement works best as a series of small experiments whose results you can see. Anthropic's prompt-engineering overview assumes you start with three things: a clear definition of success, some way to test against it empirically, and a first draft prompt you want to improve. Each revision is then a hypothesis, such as "adding the audience will fix the tone", and the next output either confirms it or does not.
Anthropic's practical advice is to start simple, add complexity only when it is needed, and test each addition to see whether it actually improves the results. Longer, more complex prompts are not always better, and using every technique at once appears on the list of common mistakes. If the core request is still vague, work on clarity first, because advanced techniques cannot make up for an unclear request.
Checkpoint 1 of 5· Check yourself
After two rounds, you realise your request never said what the output is for or who will read it. What should the next revision focus on?
Anthropic's decision framework starts with clarity. More advanced techniques come later, and only if a clear request still falls short.
“Is your request clear and explicit? If no, work on clarity first”Source: claude.com
2.Decide what 'better' means before the next round
You can only tell whether a revision helped if you know what a good answer looks like. Anthropic's evaluation guidance says good success criteria are specific, meaning they clearly define what you want to achieve, and measurable, using numbers or well-defined qualitative scales such as a 1-to-5 coherence rating. "A good supplier review" is not a criterion. "Every risk rating cites a figure from the file, and the summary fits on one page" is.
Most real tasks need more than one criterion, and the guidance suggests several questions to ask. How well does the output do the task, including on rare or difficult inputs? How similar are the answers to similar inputs? Does the response address the question directly and in a logical order? Does the style suit the audience? How well does it use the context it was given? A finance commentary that reads beautifully but quotes wrong numbers passes on style and fails on accuracy. If you name both criteria in advance, fluent prose cannot hide the error.
Anthropic's quick evaluation tips for prompts say the same thing in everyday terms. Does the output match your specific requirements, and is the format consistent across multiple attempts?
Checkpoint 2 of 5· Check yourself
Which of these is the most useful success criterion for judging whether a revised meeting-summary prompt is better?
Good criteria are specific and measurable. You can check the third option against any output, while the others leave 'better' undefined.
“Specific: Clearly define what you want to achieve.”Source: platform.claude.com
3.Test on more than the input you tuned it on
A prompt refined over many rounds against one document can end up working only for that document. Wording that compensates for one supplier's file layout, one customer's tone or one report's quirks may not hold up on the next input. Before anyone else relies on the prompt, run it on a range of real inputs.
Anthropic's evaluation guidance says to design evals that mirror the inputs your real task actually receives, and not to forget edge cases: irrelevant or missing input data, very long inputs, and ambiguous cases where even people would disagree. For a team prompt, that means trying it on typical files, unusual files and a deliberately incomplete one, and checking each output against the criteria you set.
Check consistency as well. Run the same prompt several times and compare the outputs to see whether the results are stable. The documentation notes that inconsistencies across outputs could indicate hallucinations. If the prompt contains examples, make sure they vary enough that Claude is not copying patterns from a single case.
Checkpoint 3 of 5· Exam question
A marketing coordinator's first attempt at a campaign brief comes back disappointing. She wants to improve the result on her next attempt. What should she do first?
Correct answer: C — Work out what is actually wrong with the answer, since a vague ask and missing material need different fixes.
- A. Length is not the variable. A longer version of an unclear request is still unclear, and padding can bury the one instruction that mattered.
- B. Resending an unchanged request repeats the conditions that produced the weak answer, so any improvement is chance rather than a fix.
- C. Correct. The causes call for opposite responses: a vague ask needs a tighter specification, missing source material needs the material supplied, and a wrong task needs restating. Changing something before knowing which it is means guessing.
- D. Splitting up helps when a request is genuinely doing too much, but applied blindly it produces four requests carrying the same original flaw.
Checkpoint 4 of 5· Check yourself
An analyst has spent many rounds refining a supplier-review prompt, always using the same supplier's file. It now produces excellent output for that file. What should the analyst do before making it the team's standard?
Success on one input does not show the prompt generalises. Testing on realistic and edge-case inputs shows whether it works across the files the team will actually use.
“Design evals that mirror your real-world task distribution.”Source: platform.claude.com
4.Lock in what works, and know when to stop
Iteration is not meant to produce the most elaborate prompt possible. Anthropic describes the best prompt as one that achieves your goals reliably with the minimum necessary structure. Once a refined prompt meets your criteria across varied inputs, stop changing it and make it reusable.
Claude's apps have persistent settings for instructions you want to apply beyond a single prompt. They act as standing context in every chat they cover:
| Setting | Where it applies | Good fit for |
|---|---|---|
| Instructions for Claude (account settings) | All of your conversations with Claude | Your preferred approaches, common terms, and general communication instructions |
| Project instructions | Only chats within that project | Project-specific context, guidelines for a particular workflow, requirements for a specific set of tasks |
Project instructions are a good home for a refined prompt tied to one workflow. They set requirements for that set of tasks without affecting unrelated chats. Anthropic's AI Fluency course notes that established context makes each new workflow better than starting fresh, so refinement you did once keeps paying off.
Checkpoint 5 of 5· Check yourself
You want your tested review guidelines to apply automatically in every chat about one procurement workflow, but not in your other conversations. Where should they go?
Project instructions cover only the chats inside that project. Account-wide instructions would also affect unrelated conversations.
“These instructions only apply to chats within that project.”Source: support.claude.com
Exam traps
Each one states something that sounds right. Open it to see what is actually true.
1.A prompt that reliably works on the file it was refined with is ready to become the team standard.Why is that wrong?
Repeated success on one input shows only that the prompt fits that input. Test it on inputs that reflect real-world variety, including edge cases, before others rely on it.
Covered in Test on more than the input you tuned it on
2.When a revision disappoints, adding several techniques at once gives the best chance of improvement.Why is that wrong?
Add complexity only when it is needed, and test each addition separately so you know which change helped. Longer prompts are not automatically better.
Covered in Change one thing, then test it
Sources
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Some ways to empirically test against those criteria”
↩︎ Change one thing, then test it“A first draft prompt you want to improve”
↩︎ Change one thing, then test it - 2.https://claude.com/blog/best-practices-for-prompt-engineeringSecondary source
“Longer, more complex prompts are NOT always better.”
↩︎ Change one thing, then test it“Is the format consistent across multiple attempts?”
↩︎ Decide what 'better' means before the next round“achieves your goals reliably with the minimum necessary structure”
↩︎ Lock in what works, and know when to stop“Start simple and add complexity only when needed. Test each addition to see if it actually improves results.”
↩︎ Exam trap 2“Start simple and add complexity only when needed. Test each addition to see if it actually improves results.”
↩︎ Prediction“Is your request clear and explicit? If no, work on clarity first”
↩︎ Checkpoint - 3.
“Specific: Clearly define what you want to achieve.”
↩︎ Decide what 'better' means before the next round“Most use cases need multidimensional evaluation along several success criteria.”
↩︎ Decide what 'better' means before the next round“Design evals that mirror your real-world task distribution.”
↩︎ Test on more than the input you tuned it on“Design evals that mirror your real-world task distribution.”
↩︎ Exam trap 1 - 4.https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-hallucinationsOfficial docs
“Inconsistencies across outputs could indicate hallucinations.”
↩︎ Test on more than the input you tuned it on - 5.https://support.claude.com/en/articles/10185728-understanding-claude-s-personalization-featuresOfficial docs
“Any instructions you add here will be applied to all of your conversations with Claude.”
↩︎ Lock in what works, and know when to stop“These instructions only apply to chats within that project.”
↩︎ Lock in what works, and know when to stop - 6.https://academy.claude.com/courses/ai-fluency-for-educators/applying-ai-fluency-to-learning-materials-and-assignmentsOfficial docs
“Established context makes each new workflow better than starting fresh”
↩︎ Lock in what works, and know when to stop