What you will be able to do
- Find the missing assumption, dependency or runbook element that would stop a receiving team from running a Claude system
- Say which handoff documentation closes a quality-regression gap: an evaluation package, not a dashboard
- Say what a prompt registry and a rollback runbook must contain so a bad prompt change can be undone
- Write the capacity, limit-response, monitoring and model-deprecation entries a receiving team needs
Key concept
Operational handoff completeness — A handoff package is complete when the receiving team can notice, decide on and fix every predictable operational event without the people who built the system. The events include a quality regression, a bad prompt edit, a limit being hit, a stuck session and a model retirement. Any event the team can see happening but cannot act on is a gap in the package.
1.What makes a handoff package incomplete
Several checks on this page use Claude Managed Agents, Anthropic's hosted service for running agents. An agent there is a saved definition: a model, a system prompt and a set of tools. A session is one running instance of an agent inside an environment, the sandbox that defines its container. A session is idle when it is waiting for input, such as a user message or a tool confirmation, and running while the agent is working. Every update to an agent produces a new immutable agent version, and a session chooses which version to run by ID. Webhooks are notifications the platform sends when a session changes state.
A handoff package is incomplete when it describes how the system works but not how to keep it working. The builders of a Claude system know things they never wrote down: which outputs count as good, which prompt text was live last Tuesday, how much traffic the organization's limits can absorb, and which model ID will stop answering next year. Once operations takes over, that knowledge leaves with the builders. The receiving team inherits prompts and tools that run fine on day one, along with every decision the builders made by instinct.
To find the gap, go through the events a running Claude system will certainly meet and ask the same question about each one: can the receiving team act on it using only the package? Five events come up again and again. Someone proposes a prompt change. A live edit goes wrong. Traffic grows. A session stalls or runs over its budget. The model is deprecated. Prompts, architecture diagrams and dashboards answer none of these on their own. Each needs an artifact the team can run, restore from or follow step by step. The reasons behind the design belong in architecture decision records. The handoff package holds what someone does at 3 a.m.
Platform facts make some of these events unavoidable. Models move through active, legacy, deprecated and retired states. Limits are enforced per organization. Managed Agents sessions keep their history until someone deletes them. A package that ignores these facts is not just thin. It is wrong about the environment the system will run in.
Checkpoint 1 of 8· Check yourself
A handoff package for a Claude Managed Agents system includes the prompts, an architecture diagram and a quality dashboard. It has no procedure for session data, because the builders assumed the platform discards a session's history when the session finishes. Why is this package wrong about its environment, not just thin?
The lesson lists session retention as a platform fact that makes an event unavoidable: Managed Agents keeps session history until someone deletes it. A package that assumes the history is discarded is wrong about the environment and gives the receiving team no retention or deletion procedure to follow.
“Session history: Kept until you delete the session.”Source: platform.claude.com
2.An evaluation package, not a quality score
An aggregate production score tells you that quality has dropped. It cannot tell you that a proposed change will make quality drop. Those are different questions, and the receiving team usually needs the second answer more. A score that refreshes hourly and turns red at a threshold only reports a regression after users have already seen it. Explaining how the score is aggregated does not change when it reports.
What closes the gap is the builders' evaluation, handed over in a form the team can rerun. Success criteria should be specific and measurable: "accurate sentiment classification", not "good performance". They become a dataset of inputs, graders that score each output (exact match where an answer has one correct form, model-based grading where it does not), and a pass threshold per criterion. The package also needs a procedure that says when to run the evaluation: before any prompt or model change ships, comparing the candidate with the version currently live. A/B comparison against an earlier version is a standard measurement method for exactly this purpose. Choosing what the dataset should cover is its own skill, covered under evaluation coverage gaps. The handoff question is only whether the evaluation leaves with the builders or stays with the system.
Prompt-style guides and recordings of hand checks miss for the same reason. They describe how the builders worked, not a bar a change has to clear. A conventions document tells an engineer how to edit a prompt. It does not tell them whether the edit passed.
Checkpoint 2 of 8· Exam question
A Claude-based contract-summarization service is handed to a support team. The package includes prompts and a dashboard showing an overall quality score, but the receiving team cannot tell whether a proposed prompt change makes summaries worse. Which documentation should be added?
Correct answer: B — An evaluation package with the dataset, grader definitions, and pass thresholds, plus steps for running it before a prompt or model change ships
- A. A guide to how the production score is aggregated and displayed, with escalation steps, only reports quality after a change is live. It gives the team no way to test a proposed prompt change beforehand, so they still cannot tell whether it makes summaries worse.
- B. The evaluation dataset, grader definitions, and pass thresholds, with a procedure to run them before a change ships, let the team score a proposed prompt or model change against a fixed baseline and see a regression before release.
- C. A prompt-engineering guide helps the team read and edit the prompt files, but it contains no test data, graders, or thresholds. Understanding the prompts does not tell the team whether an edit makes summaries worse.
- D. Five hand-picked contracts checked by eye are a small anecdotal sample with no graders or pass thresholds. Different reviewers will judge them differently, so the checks do not give a consistent verdict on whether a change degrades summaries.
Sources6
3.Prompt version ownership and rollback
If prompt text is edited in place in production, nobody can say what was running before an incident. When the system prompt sits in a configuration service and engineers overwrite it directly, a sudden spike in refusals leaves the receiving team stuck. They can see that something changed, but not what to change it back to. A refusal-rate alert would spot the spike sooner. It still would not recover the previous text.
The fix is to make each prompt version immutable and addressable. In Claude Managed Agents, every update to an agent creates a new version, and callers choose a version by ID. Rolling back means pointing callers at the old ID, with no rebuild or redeploy. Within a running session the configured system prompt does not change, so the version a session started on is the version it runs with. The same design works anywhere: a registry that stores each saved version under a release identifier tied to its deployment. A wiki page that the editing engineer overwrites fails for the same reason the production edit did, because it keeps only the newest text.
Ownership is part of the registry. Moving prompts out of code does not remove review. The approval now covers a version number in configuration instead of a code diff, so the package has to name who approves a change to the live version. The rollback runbook then gives three facts: which earlier version to restore, who may trigger the restore, and which checks to run afterwards. The last of these is normally the evaluation package from the previous section, scored against the restored version.
It changes when edits happen, not what is recorded about them. An edit made at 11 a.m. still overwrites the previous text. After a refusal spike nobody can identify the prior version or restore it. Versioning and a rollback runbook fix that. A change window does not.
Checkpoint 3 of 8· Check yourself
After a refusal spike, the on-call engineer rolls back by pointing callers at the earlier agent version ID. One Managed Agents session started on the faulty version an hour ago and is still open. What happens to that session?
The configured system prompt does not change during a running session. A session runs with the version it started on. Rolling back affects only sessions created after callers point at the old version ID, so the runbook has to say how to handle sessions that are already running. Mid-session updates can change only tools and MCP servers, not the system prompt.
“The agent’s configured system field is fixed for the session’s lifetime.”Source: platform.claude.com
Checkpoint 4 of 8· Exam question
A product team stores a Claude system prompt in a configuration service and edits it directly in production. Operations is taking over the service. After a recent edit, refusals spiked and nobody could say which text was live before it. Which two items should the handoff package add? Choose two.(Select 2)
Correct answers: B, D — A prompt registry that stores each saved version under a release identifier linked to its deployment, with a named approver for live changes; A rollback runbook that names the previous prompt version to restore, who can trigger the restore, and which checks to run afterwards
- A. Monitoring would flag a refusal spike sooner, but the spike was already noticed. The gap is that nobody could say which prompt text was live before the edit, and a dashboard records no prompt text and cannot restore any.
- B. A registry that keeps every saved version under a release identifier tied to deployments lets operations say which text was live before an edit. The named approver gives live prompt changes a clear owner.
- C. The page is overwritten after each change, so it holds only the current text and the previous version is lost. It also depends on a person remembering to update it, so it cannot show what was live before the edit.
- D. A rollback runbook that names the version to restore, who may trigger it and the post-restore checks lets operations reverse a bad edit during an incident without waiting for the original authors.
- E. Restricting edits to business hours with someone on call adds supervision, but edits still overwrite the live prompt in place. It creates no version history and no way to restore earlier text, so the missing information stays missing.
4.Capacity assumptions and limit responses
A package that leaves out capacity has quietly assumed the organization's limits will absorb whatever traffic comes. The Claude API has two kinds of limit, and they fail in different ways. Rate limits cap how many requests and tokens an organization can send in a period of time. Spend limits cap monthly cost. Both apply to the whole organization, so a new region sharing the organization competes with existing traffic for the same headroom. The limits are also ceilings, not reserved capacity. Rate limiting uses a token bucket that refills continuously, and short bursts can trip it even when the per-minute average looks safe.
For a launch that doubles peak traffic, two documents close the rate-limit gap. The first is a load model: assumed peak requests and tokens per minute, set beside the organization's current limits from the Console's Rate limits page, plus the named owner who requests an increase. The second is a runbook entry for 429 responses in the client. It covers backoff that follows retry-after, keeping requests queued instead of firing them all at once, and a defined way to degrade service when headroom runs out. The same principle applies inside agents: wide parallel fan-outs can hit rate limits, so break the work into smaller batches. Retry mechanics are covered under production reliability controls.
Two plausible additions miss the point. A cost forecast for doubled traffic covers the spend limit, which is real but is not the limit the peak will hit first. A sizing sheet of context windows and output tokens is about the size of a single request, not throughput.
The runbook also has to tell apart responses that look the same. When the organization reaches its tier's monthly spend cap, the API returns HTTP 429 with error type rate_limit_error, but the response has no retry-after header and error.details.error_code is enforced_spend_limit_reached. A client that treats every 429 as a rate limit will keep retrying, and every retry will fail until the month resets or the organization moves to a higher tier.
{
"type": "error",
"error": {
"type": "rate_limit_error",
"message": "You have reached your API usage limits: your organization has crossed its monthly API usage threshold, set based on your organization's API tier. You will regain access on 2026-09-01 at 00:00 UTC.",
"details": { "error_code": "enforced_spend_limit_reached" }
},
"request_id": "req_018EeWyXxfu5pfWkrYcMdjWG"
}| Response | Retry? | What restores access |
|---|---|---|
| Rate limit: 429 rate_limit_error | Yes, back off | Capacity refills over time |
| Tier spend cap: 429 with enforced_spend_limit_reached, no retry-after | No, retries fail until access resumes | Next month at 00:00 UTC, or a higher tier |
| Self-set spend limit: 400 invalid_request_error | No | Raise or remove the limit in the Console |
Checkpoint 5 of 8· Match them up
Your 429 runbook entry needs a different action for each limit response the client can receive. Match each response to the action the runbook should specify.
Tap a term, then the definition that fits it.
The two 429s have the same error type. Only error_code and the missing retry-after header show that the second one is the tier spend cap, and retrying it keeps failing until access resumes. A spend limit the organization set itself returns a 400 instead, and access comes back when someone changes that limit.
“The error type is rate_limit_error, the same as for a rate limit, but the response has no retry-after header.”Source: platform.claude.com
Checkpoint 6 of 8· Exam question
A retail company is about to double traffic to a Claude-powered assistant by launching in a new region. The handoff package to site reliability describes the prompts and tools but is silent on capacity. Peak-hour requests are expected to hit API rate limits. Which two additions to the package close the gap? Choose two.(Select 2)
Correct answers: A, C — A load model listing the assumed peak requests and tokens per minute beside the organization's current limits, and the owner who requests an increase; A runbook entry covering 429 handling in the client: backoff that follows the retry-after header, a bounded queue, and a defined degrade path
- A. Setting the assumed peak requests and tokens per minute against the organization's current limits, and naming the owner who requests an increase, gives site reliability the missing capacity assumption and a named owner for acting on it.
- B. Context window and maximum output tokens are per-request ceilings. The expected problem is the per-minute request and token rate limits at peak, which this sheet does not size or compare to the organization's limits.
- C. Throttled calls return 429 responses. A runbook entry for retrying with backoff that follows the retry-after header, queueing within a bound, and degrading gracefully tells operators how the system behaves when peak traffic hits the limits.
- D. Trimming prompts can lower tokens per request, but it is an optimization rather than a capacity statement. It does not compare the doubled load to the limits, name an owner for raising them, or say what happens when requests are throttled.
- E. Spend and budget approval concern cost. Rate limits cap throughput per minute regardless of budget, so this forecast neither shows whether peak load fits the limits nor describes how to handle throttling.
5.Who answers each operational signal
An alert with no named responder is a gap, even when the alert works perfectly. For a Claude Managed Agents deployment, the package should list each webhook the team subscribes to and say who acts on it. It should also record where the webhook signing secret lives, because that secret is shown only once at creation. Choosing which metrics to instrument is a separate topic, covered under instrumentation and monitoring gaps. The handoff question is whether each signal leads to a decision someone owns.
| Signal | What it means | Decision the runbook assigns |
|---|---|---|
| session.status_idled | Session went idle, possibly waiting on approval | Check why it went idle and answer pending tool calls |
| session.budget_reached | Session hit its spend cap | Raise the cap or leave the work paused |
| session.status_terminated | Session ended | Investigate the unrecoverable error |
| deployment_run.failed | Scheduled run failed to start a session | Restart or escalate the schedule |
| vault_credential.refresh_failed | MCP OAuth credential cannot refresh | Re-authorize the credential |
Webhooks alone are not enough coverage. A delivery can be missed, so the runbook also needs a scheduled job that lists sessions and checks the idle ones. Self-hosted Agent SDK deployments have a similar silent failure. When the SDK cannot deliver a transcript batch to its session store, it drops the batch, emits a mirror_error message and carries on. If the team relies on those transcripts, something has to alert on that message.
Retention belongs in the same section of the package. Managed Agents keeps session history until the session is deleted. Files a session produced are deleted along with it, while uploaded files, memory stores and vaults have to be deleted separately. A team that cleans up sessions without first copying events to its own logs loses the record it would need for an incident review. Archiving instead of deleting keeps the record for audit.
Checkpoint 7 of 8· Check yourself
The receiving team's runbook for a Managed Agents support deployment subscribes to session.status_idled, session.budget_reached and session.status_terminated. It names an owner for each webhook, and the signing secret is stored in a secrets manager. What else must the package include so that no session needing attention goes unnoticed?
A webhook delivery can be missed, so the webhooks alone do not cover every session. The runbook also needs a scheduled job that lists sessions and checks the idle ones. Archiving on idle is wrong because an idle session may be waiting for someone to approve a tool call.
“You can miss a webhook delivery, so also list your sessions on a schedule and check the idle ones.”Source: platform.claude.com
6.A model-deprecation plan
Every Claude system is built on a model that will eventually retire. A handoff that hard-codes a model ID without a plan for that day leaves the receiving team with an outage on a fixed date. A deprecated model still works, but Anthropic names a recommended replacement and sets a retirement date. After that date, requests to the model fail. The notice goes out by email and in the documentation, so the package must say whose inbox receives it. A notice sent to a builder who has moved teams means the receiving team never sees it.
The deprecation plan should list each model ID the system calls, the platform it runs on, the owner who watches lifecycle notices, and the steps for moving to the replacement. Platform matters because partner-operated platforms such as Amazon Bedrock set their own retirement schedules, so the same model can have different dates on different platforms. The migration steps reuse the earlier artifacts. The evaluation package scores the replacement model against the current one, the prompt registry records the version paired with the new model, and the rollback runbook covers the switch back. How to carry out the migration itself is covered under model version migration.
Sessions that are already open need their own line in the plan. After a Managed Agents session is created, only its tools and MCP servers can change. Its model cannot be updated, so an open session keeps running on the model it started with. To run on a different model, a session has to be created that way from the start, for example on a new agent version.
Checkpoint 8 of 8· Check yourself
A reviewer is checking the migration steps in a model-deprecation plan for a Claude Managed Agents system before the old model's retirement date. Which step will not work as written?
Only a session's tools and MCP servers can change after it is created, so an open session keeps running on the model it started with. Sessions still open on the deprecated model have to finish, or be recreated on the new agent version, before the retirement date, because requests to the old model fail after that date.
“Only the agent’s tools and mcp_servers can change after a session is created.”Source: platform.claude.com
Sources
(8)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Applications relying on Anthropic models may need occasional updates to keep working.”
↩︎ What makes a handoff package incomplete“Once a model is deprecated, migrate all usage to a suitable replacement before the retirement date.”
↩︎ A model-deprecation plan“Impacted customers will always be notified by email and in the documentation.”
↩︎ A model-deprecation plan“Requests to models past the retirement date will fail.”
↩︎ A model-deprecation plan“Applications relying on Anthropic models may need occasional updates to keep working.”
↩︎ Key concept - 2.https://platform.claude.com/docs/en/api/rate-limitsOfficial docs
“Limits are set at the organization level.”
↩︎ What makes a handoff package incomplete“Rate limits set the maximum number of API requests an organization can make over a defined period of time.”
↩︎ Capacity assumptions and limit responses“Short bursts of requests can exceed the limit and trigger rate limit errors.”
↩︎ Capacity assumptions and limit responses“The error type is rate_limit_error, the same as for a rate limit, but the response has no retry-after header.”
↩︎ Capacity assumptions and limit responses“Retrying, including the SDK’s automatic retries, fails until access resumes.”
↩︎ Exam trap 2“All limits described here represent maximum allowed usage, not guaranteed minimums.”
↩︎ Exam trap 3 - 3.
“Session history: Kept until you delete the session.”
↩︎ What makes a handoff package incomplete“You can miss a webhook delivery, so also list your sessions on a schedule and check the idle ones.”
↩︎ Who answers each operational signal“To keep a session’s events, copy them to your own logs with the events API before you delete it.”
↩︎ Who answers each operational signal“You can miss a webhook delivery, so also list your sessions on a schedule and check the idle ones.”
↩︎ Exam trap 4 - 4.
“Agent is waiting for input, including user messages or tool confirmations.”
↩︎ What makes a handoff package incomplete“The agent’s configured system field is fixed for the session’s lifetime.”
↩︎ Prompt version ownership and rollback“Only the agent’s tools and mcp_servers can change after a session is created.”
↩︎ A model-deprecation plan - 5.https://platform.claude.com/cookbook/managed-agents-cma-prompt-versioning-and-rollbackSecondary source
“Every agents.update produces a new immutable version, and sessions choose which version to use by ID.”
↩︎ What makes a handoff package incomplete“Every agents.update produces a new immutable version, and sessions choose which version to use by ID.”
↩︎ Prompt version ownership and rollback“If something goes wrong, pointing callers back at the old version is all it takes to roll back.”
↩︎ Prompt version ownership and rollback“you're approving a version number in config rather than a code diff”
↩︎ Prompt version ownership and rollback - 6.
“Building a successful LLM-based application starts with clearly defining your success criteria and then designing evaluations to measure performance against them.”
↩︎ An evaluation package, not a quality score“Instead of “good performance,” specify “accurate sentiment classification.””
↩︎ An evaluation package, not a quality score“A/B testing: Compare performance against a baseline model or earlier version.”
↩︎ An evaluation package, not a quality score“A/B testing: Compare performance against a baseline model or earlier version.”
↩︎ Exam trap 1 - 7.https://code.claude.com/docs/en/agent-sdk/hostingOfficial docs
“Break work into smaller batches rather than issuing one wide dispatch.”
↩︎ Capacity assumptions and limit responses“Alert on these if store durability matters.”
↩︎ Who answers each operational signal - 8.
“it's the signal a supervisor process needs to decide, per session, whether to raise the cap or leave the work paused”
↩︎ Who answers each operational signal“archive keeps the record around for audit and retrieval”
↩︎ Who answers each operational signal