What you will be able to do
- Classify a failed or stopped Claude call as a rate limit, spend cap, overload, refusal, or context-window stop, and choose the recovery that fits it
- Design retries that honour retry-after, back off exponentially and spread a fleet's retries with jitter
- Explain when a circuit breaker and a degraded reply beat more retries during a sustained overload
- Configure cross-model fallback for refusals, and plan cross-region and cross-platform fallbacks around what each platform actually supports
Key concept
Classify before you recover — Every failed or incomplete Claude call has a cause: a throttle, a spend cap, server overload, a safety refusal or a truncation. Each cause has its own correct recovery. If you apply one generic retry to all of them, you will wait on failures that can never succeed, put more load on an overloaded service, and miss refusals entirely.
1.Three different kinds of "it didn't work"
A production Claude integration fails in three different ways, and each one needs a different response. The first is an HTTP error: the request was throttled, rejected or not served. The second is a refusal: the request succeeded, but a safety classifier declined it. The third is a stop that truncates the output, such as model_context_window_exceeded. All three can look like "the call failed" to a caller, which is why one catch-all retry loop is the most common reliability bug in Claude integrations.
Errors and stop reasons arrive through different channels. An error carries a non-200 status and a typed body, and the SDK raises it as an exception. A stop reason is a field on a successful response, so no exception handler ever sees it. Code that only wraps messages.create in try/except handles the first channel and is blind to the second.
| Signal | What it means | Recovery |
|---|---|---|
| 429 rate_limit_error with retry-after | An RPM, ITPM or OTPM limit was exceeded | Wait the retry-after seconds, then back off |
| 429 rate_limit_error, no retry-after, error_code enforced_spend_limit_reached | The usage tier's monthly spend cap was reached | Do not retry; it keeps failing until access resumes or the tier is raised |
| 400 invalid_request_error about specified usage limits | A spend limit you set yourself was reached | Raise or remove the limit |
| 500 api_error | An unexpected internal error | Retry with exponential backoff; quote the request ID if it persists |
| 529 overloaded_error | The API is temporarily overloaded | Back off; if it is sustained, trip a circuit breaker and degrade |
| 504 timeout_error | The request timed out while processing | Use streaming for long requests rather than retrying the same call |
The spend-cap row is the one that catches teams out. It has the same rate_limit_error type as an ordinary throttle, but no retry-after header, and retrying cannot fix it. Branch on error.details.error_code to tell the two apart. A 429 can also appear while your dashboard shows usage below the limits: a sharp increase in traffic triggers ramp limits. Throughput planning for that case is covered in rate limits and ramp-up.
| stop_reason | What it means | Recovery |
|---|---|---|
| refusal | Claude declined to respond | Read stop_details and retry on a fallback model |
| max_tokens | The response reached your max_tokens limit | Raise max_tokens or continue the response |
| model_context_window_exceeded | The response filled the model's context window | Treat the response as truncated |
| pause_turn | A server-tool loop reached its iteration limit | Send the assistant content back to continue |
A context-window stop is not a throttle and not a refusal. Re-sending the request to escape a rate limit misreads it, and so does routing it through refusal fallback. The output is truncated, so the application must not present it as a complete answer.
Checkpoint 1 of 9· Match them up
A production integration runs into each of these signals. Match each one to the recovery it calls for.
Tap a term, then the definition that fits it.
Each signal has its own cause, so one catch-all retry fails here. A spend-cap 429 has the same error type as a throttle but no retry-after header, and retrying cannot fix it. Refusals and context-window stops come back as successful responses: one needs a fallback model, the other is a truncated output.
“The error type is rate_limit_error, the same as for a rate limit, but the response has no retry-after header.”Source: platform.claude.com
2.Retries that do not recreate the burst
The SDK already retries for you. Connection errors, rate limits and 5xx responses are retried with exponential backoff, twice by default, and the SDK honours retry-after when the header is present. max_retries tunes or disables this. Any retry loop you write sits on top of the SDK's, so three application retries on top of two SDK retries can mean many more attempts per user request than anyone intended.
Rate limiting runs on a token bucket that refills continuously, not on a counter that resets at fixed intervals. So the problem is the shape of your traffic, not its total. A limit of 60 RPM can be enforced as one request per second, and a burst exceeds it even when the minute's total would fit. Fixed retry intervals keep a fleet synchronized: every worker that failed together retries together, and each retry round becomes a new burst.
A retry policy for a fleet therefore has three parts:
- On a 429 with
retry-after, wait that many seconds first. The server is telling you when capacity returns. - Grow each later sleep exponentially, so a still-throttled worker backs off further instead of hammering at a constant rate.
- Add a random offset (jitter) to each worker's delay, so the fleet's retries spread across the refill window instead of landing together.
Raising the retry cap or moving to a flat 5-second sleep fixes none of this, because the lockstep stays. Routing retries to a larger model to borrow its quota only moves the burst to another bucket.
Timeouts need their own design. A gateway timeout shorter than a long generation turns every slow request into a timeout, then a retry, then another full generation, and the request still fails. For long outputs, stream. If you only want the finished message, the SDK can consume the stream and return the same object a non-streaming call would.
client = anthropic.Anthropic()
with client.messages.stream(
max_tokens=128000,
messages=[{"role": "user", "content": "Write a detailed analysis..."}],
model="claude-sonnet-5",
) as stream:
message = stream.get_final_message()
print(next(block.text for block in message.content if block.type == "text"))Checkpoint 2 of 9· Check yourself
A report-writing feature asks for very long outputs. The gateway in front of the app times out before generation finishes, so the app retries, and every retry times out the same way. What fixes this?
A timeout shorter than the generation means every retry runs the full generation again and hits the same timeout. More retries, backoff or another model don't change that. Streaming does, and the SDK can still return the complete message as a non-streaming call would.
“For long-running Messages API requests, use the streaming Messages API.”Source: platform.claude.com
Checkpoint 3 of 9· Exam question
Birch Games runs a nightly enrichment job in production that tags player-support transcripts with Claude, using a fleet of 200 workers. At the top of each hour, a burst of 429 responses arrives, each carrying a retry-after value. The workers use a fixed 2-second retry loop. Traffic graphs show that loop re-creating the burst every 2 seconds, with each spike nearly as tall as the first. Request logs show almost all 200 workers sending within the same fraction of a second. The team already waits out the retry-after value on the first rejection. Output quality was validated on the current model, and company policy requires a formal re-evaluation before any model change. The job has a long overnight window, so extra delay per request is acceptable. What is the best fix?
Correct answer: B — Wait out the retry-after value on the first 429, then grow each later sleep exponentially, with a random offset added to each worker's delay.
- A. Workers that all receive the same retry-after and then sleep the same flat 5 seconds stay synchronized. The fleet still retries together and recreates the burst, only less often.
- B. The first wait respects the server's own signal about when capacity returns. Exponential growth with a random offset per worker then spreads later retries out instead of letting 200 workers return at once.
- C. A different model changes cost and quality but not the retry pattern. The fixed 2-second sleep still keeps the workers synchronized, so the same wave would arrive at the new model's quota.
- D. Waiting out retry-after once does not break the synchronization. Workers then retry together every 2 seconds, and a higher cap gives them more synchronized rounds of load during the burst.
3.Circuit breakers for sustained overload
Backoff is built for brief failures. A 529 overloaded_error means the API is temporarily overloaded, and during a provider-wide incident "temporarily" can mean minutes. In that window, every retry with backoff ties up a worker thread for the whole backoff schedule. Multiply that by the SDK's retries and your own, and the thread pool drains. Endpoints that never call Claude then start failing too, because nothing is left to serve them.
A circuit breaker sits in front of the Claude client and tracks the recent error rate. While it is closed, calls pass through. When failures cross a threshold, it opens: calls fail immediately with a degraded reply, and no request leaves the service. After a cool-down it goes half-open and lets a few trial requests through. If they succeed it closes; if they fail it opens again. Failing fast protects the rest of the service, and it also stops your fleet from adding load to a provider that is already overloaded.
Other fixes look reasonable but make things worse. A larger thread pool and a longer timeout just let more requests wait on the same outage. More retries add load to an overloaded service and push latency higher. Restarting pods on every 529 treats a provider-side condition as a local fault and throws away healthy capacity. When the breaker is open, lower max_retries on the SDK client as well, so its built-in retries do not quietly rebuild the waiting you just removed.
Checkpoint 4 of 9· Check yourself
During a provider-wide incident, a sustained run of 529 overloaded_error responses hits a team's service. They have already removed their own application-level retry loop, yet each user request still holds a worker thread through several backed-off attempts before it fails. What explains this, and what should they change?
The SDK has built-in retries with backoff for 5xx errors, so removing your own loop still leaves the SDK's attempts tying up threads. Failing fast during sustained overload means lowering max_retries as well. A bigger pool or a longer timeout only lets more requests wait on the same outage.
“The official SDK automatically retries transient failures (such as connection errors, rate limits, and 5xx server errors) with exponential backoff, twice by default”Source: platform.claude.com
Checkpoint 5 of 9· Exam question
Granite Insurance runs a claims-assistant chat backend on a shared worker thread pool that also serves its policy-lookup and claims-status endpoints. During a provider-wide incident, its Claude calls return 529 responses. Each request retries 3 times with backoff, and traces show nearly every call still fails after the final attempt, so threads stay occupied for the whole retry window. p99 latency has reached 90 seconds, and the exhausted pool now takes down the unrelated endpoints. The provider's status page gives no recovery estimate. Product owners accept a short, simplified reply in chat during outages. On-call engineers want normal behavior to resume automatically once Claude recovers, without manual intervention or a full traffic surge at the first sign of recovery. What is the best change?
Correct answer: B — Wrap Claude calls in a circuit breaker that opens above an error-rate threshold, fails fast to a degraded reply, and probes with trials.
- A. More threads and longer waits let slow calls accumulate and delay recovery of the other endpoints; capacity is not the underlying problem.
- B. Failing fast releases threads immediately and isolates the blast radius, while half-open probes detect recovery without sending full traffic.
- C. Extra retries hold threads longer and add load to an overloaded service, worsening both latency and the shared-pool exhaustion.
- D. Restarting pods does nothing about upstream overload and adds churn; detection without a fail-fast path leaves requests to hang.
Sources3
4.Degrading gracefully instead of failing whole
Graceful degradation means deciding in advance which parts of a feature are essential and which are optional. Take contract review. Clause extraction is the core output and a risk narrative is an extra. If one sustained overload fails both and the user sees an error page, the design has treated the optional step as mandatory.
Honest degraded replies include:
- Returning the core output that did succeed, with a notice that the optional part is delayed, and queuing that part for background retry.
- Serving the most recent cached result for unchanged inputs, labelled as stale with its timestamp.
- Answering from a fallback model and recording that it did so, so quality can be checked later.
The label is what keeps these honest. Unlabelled stale data or silent substitutes become correctness bugs.
The tempting wrong options pretend analysis happened when it didn't. Auto-approving contracts that have no analysis turns an availability problem into a compliance problem. A keyword template that declares "no material risks found" is a fabricated result shown with the same confidence as a real one. A generic error page is honest but throws away the clause extraction the user could have had. The same rule covers partial model output: a response interrupted mid-stream is incomplete, and it should never be shown as an answer.
Checkpoint 6 of 9· Check yourself
Clause extraction for a contract has come back complete. The optional risk-narrative call streamed half a narrative, then stopped with stop_reason refusal. What is the honest degraded reply to the user?
Partial output from an interrupted stream is incomplete and must not be shown as an answer, even with a caution label. The essential clause extraction succeeded, so serve it with a clear label and queue the optional part. A template summary would make up a result, and an error page would throw away output that worked.
“A refusal can arrive before any output, or mid-stream after partial output. In either case, treat any partial output as incomplete and discard it.”Source: platform.claude.com
Checkpoint 7 of 9· Exam question
Heron Labs runs contract-review software for its in-house legal customers. Each upload triggers clause extraction, which is the core deliverable, followed by an optional risk narrative that reviewers read when time allows. During a sustained period of 529 overloaded responses, both steps fail and users see an error page. Reviewers have said clause extraction alone lets them start work, and many re-open documents that have not been edited since their last analysis. Counsel has ruled that no contract may advance in the approval workflow, or be described as low risk, without output the system actually produced from that document. Support also reports that users repeatedly hit retry on the error page. What should the team do to degrade gracefully? Select TWO.(Select 2)
Correct answers: A, C — Return the extracted clauses with a notice that the narrative is delayed, and queue the narrative for background retry.; Serve the most recent cached analysis for unchanged documents, labelled stale with its timestamp.
- A. Separating core from optional work keeps the valuable result flowing while the nonessential step catches up later.
- B. A blanket error removes all value for users and invites manual retries, so it neither degrades gracefully nor reduces load.
- C. A clearly marked stale result is a reasonable degraded answer when the document has not changed since it was analyzed.
- D. Skipping review turns an availability problem into a risk decision nobody made, which is unacceptable for contracts.
- E. A fabricated all-clear misleads reviewers; a degraded mode must never assert findings the system did not produce.
Sources4
5.Cross-model fallback for refusals
Classifier refusals arrive as normal responses. The status is 200, stop_reason is "refusal", content is usually empty, and stop_details.category names the policy area: cyber, bio, frontier_llm, reasoning_extraction or general_harms. The usual recovery is to send the same request to another model. Retrying the same model is the wrong move, because it usually refuses again. Branch on stop_reason itself, not on category or explanation; both of those can be null, and the explanation text is not stable.
There are three ways to set up fallback, and which ones you can use depends on platform and model. Server-side fallback (beta) is the simplest: set fallbacks to "default" with the beta header, and the API retries on the model Anthropic recommends for that refusal category, all inside one call. SDK middleware works on any platform and walks your fallback list in order. A manual retry gives full control, and you redeem fallback credit so you don't pay the prompt-cache write twice. For multi-turn conversations, stay on the model that accepted rather than switching back.
client = Anthropic()
response = client.beta.messages.create(
model="claude-fable-5",
max_tokens=1024,
messages=[{"role": "user", "content": "Hello, Claude"}],
fallbacks="default",
betas=["server-side-fallback-2026-07-01"],
)
print(response.model)Fallback has several edge cases:
reasoning_extractionhas no recommended fallback model, because the prompt itself is the problem. Ask for a short explanation instead of the model's internal reasoning.- Claude Haiku 5.5 has no server-side fallback and no fallback credit.
- The
fallbacksparameter does not carry over to model calls made from inside tool execution, so subagents need their own. - Every request path needs fallback configured, including retry handlers and background workers.
- A refusal still counts against your rate limits, so a refusal storm also uses up throughput.
Fallback paths also need their own monitoring. Emit one event per refusal and one per fallback-served response (a fallback_message entry in usage.iterations marks the latter), and alert on the gap between the two counts. Track output quality on the fallback path separately too, because high availability through a weaker model can hide an accuracy drop that nobody sees. Building those signals into dashboards is covered in production monitoring and alerting.
Checkpoint 8 of 9· Fill the gap
This call should send a refused request to Anthropic's recommended fallback model within the same API call. Which parameter name completes it?
client = Anthropic()
response = client.beta.messages.create(
model="claude-fable-5",
max_tokens=1024,
messages=[{"role": "user", "content": "Hello, Claude"}],
? ="default",
betas=["server-side-fallback-2026-07-01"],
)
print(response.model)Server-side fallback is turned on by setting fallbacks to "default" and sending the server-side-fallback beta header. The API then reruns a refused request on the model Anthropic recommends for that refusal category. max_retries only controls the SDK's retries of transient errors on the same model, and a refusal is not one of those errors.
Source: platform.claude.comSources4
6.Cross-region and cross-platform fallback, and feature parity
Cross-region fallback is often handled by choosing the right endpoint type rather than writing your own failover. On Amazon Bedrock, global endpoints route requests dynamically to regions with available capacity and carry no premium. Regional endpoints pin traffic to a geography, which data-residency and compliance requirements may demand, and they cost 10 percent more. If residency is flexible, the global endpoint gives you regional failover for free. If it is not, any fallback has to stay inside the permitted regions.
Cross-platform fallback, such as sending traffic from the Claude API to a cloud provider during an incident, only works for features that exist on both sides. The request formats differ: on Google Cloud's Agent Platform the model goes in the endpoint URL, not the body, and model availability varies by region. Limits differ too: Bedrock caps request payloads at 20 MB, against 128 MB for the first-party Messages API. Model line-ups differ, and lifecycle dates on partner platforms are set by the partner. A fallback target that lacks the model, the payload size or the feature your request depends on fails exactly when you need it most.
| Platform | Server-side fallback | Models listed |
|---|---|---|
| Claude API | Yes (beta) | Fable 5.2, 5.1, 5; Mythos 5.2, 5.1; Opus 5.5, 5; Sonnet 5.5; Haiku 5.5 |
| Claude Models on Google Cloud (Anthropic managed) | Yes (beta) | Fable 5.2, 5.1, 5; Mythos 5.2; Opus 5.5, 5; Sonnet 5.5; Haiku 5.5 |
| Claude Platform on AWS | No; use SDK middleware or manual retry | Fable 5.2, 5.1, 5; Opus 5.5, 5; Sonnet 5.5; Haiku 5.5 |
| Amazon Bedrock | No; use SDK middleware or manual retry | Fable 5.1, 5; Mythos 5.1; Opus 5.5, 5; Sonnet 5.5; Haiku 5.5 |
| Microsoft Foundry (hosted on Azure) | No; use SDK middleware or manual retry | Opus 5.5, 5; Sonnet 5.5; Haiku 5.5 |
Batches follow different rules again. A batch request that sets fallbacks comes back as an errored item. A refused batch item comes back as succeeded with stop_reason: "refusal", so a batch pipeline has to scan its results for refusals and resubmit them on a fallback model. Treat parity as something you test: for each fallback target, confirm that the model, the parameters and the request size your primary path uses are all accepted there.
Checkpoint 9 of 9· Check yourself
Your app runs Claude Opus 5.5 on the Claude API, and server-side fallback (fallbacks="default") handles its refusals. During an incident, the failover plan moves the same request code to Amazon Bedrock. What does the Bedrock path need so that refused requests still reach a fallback model?
Server-side fallback is offered only on the Claude API and on Claude Models on Google Cloud (Anthropic managed). On Bedrock and Claude Platform on AWS, refusal fallback has to come from the SDK middleware or a manual retry. Global endpoints route requests by capacity and do not handle refusals.
“The simplest setup, in beta on the Claude API and on Claude Models on Google Cloud (Anthropic managed): set fallbacks to "default"”Source: platform.claude.com
Sources
(6)
Every claim above is drawn from one of these pages, quoted as it was written on the date shown.
- 1.
“Unlike errors, which indicate failures in processing your request, stop_reason tells you why Claude completed its response generation.”
↩︎ Three different kinds of "it didn't work"“Check this field to decide whether to use the response as-is, continue the conversation, retry, or fall back to another model.”
↩︎ Key concept - 2.https://platform.claude.com/docs/en/api/rate-limitsOfficial docs
“Use it to tell this response apart from a rate limit.”
↩︎ Three different kinds of "it didn't work"“your capacity is continuously replenished up to your maximum limit, rather than being reset at fixed intervals.”
↩︎ Retries that do not recreate the burst“When the response has a retry-after header, wait that many seconds before retrying.”
↩︎ Retries that do not recreate the burst“The error type is rate_limit_error, the same as for a rate limit, but the response has no retry-after header.”
↩︎ Checkpoint“Short bursts of requests can exceed the limit and trigger rate limit errors.”
↩︎ Prediction - 3.https://platform.claude.com/docs/en/api/errorsOfficial docs
“A sharp increase in usage can also cause a 429 even while you’re under your rate limits”
↩︎ Three different kinds of "it didn't work"“with exponential backoff, twice by default, honoring the retry-after header when present”
↩︎ Retries that do not recreate the burst“For long-running Messages API requests, use the streaming Messages API.”
↩︎ Retries that do not recreate the burst“529 - overloaded_error: The API is temporarily overloaded.”
↩︎ Circuit breakers for sustained overload“The SDK client accepts max_retries to configure or disable this behavior.”
↩︎ Circuit breakers for sustained overload“A tier spend-cap 429 has no retry-after header and keeps failing until access resumes”
↩︎ Exam trap 1“The official SDK automatically retries transient failures (such as connection errors, rate limits, and 5xx server errors) with exponential backoff, twice by default”
↩︎ Checkpoint - 4.
“A refusal can arrive before any output, or mid-stream after partial output. In either case, treat any partial output as incomplete and discard it.”
↩︎ Degrading gracefully instead of failing whole“The fallbacks parameter does not carry over to model calls made from inside tool execution.”
↩︎ Cross-model fallback for refusals“This category has no recommended fallback model, so change the prompt rather than retry the request.”
↩︎ Cross-model fallback for refusals“For multi-turn conversations, keep using the fallback model for subsequent turns rather than switching back.”
↩︎ Cross-model fallback for refusals“The request still counts against your rate limits.”
↩︎ Cross-model fallback for refusals“Server-side fallback is not available for batches”
↩︎ Cross-region and cross-platform fallback, and feature parity“Re-sending a refused request to the same model usually earns another refusal.”
↩︎ Exam trap 2“Instrument refusals as their own signal.”
↩︎ Exam trap 3“Claude Haiku 5.5 has no server-side fallback, so for it use client-side fallback with the SDK middleware or write the retry yourself.”
↩︎ Exam trap 4“A refusal is an HTTP 200, so monitoring built on error rates or 5xx responses never sees it.”
↩︎ Prediction“The simplest setup, in beta on the Claude API and on Claude Models on Google Cloud (Anthropic managed): set fallbacks to "default"”
↩︎ Checkpoint - 5.
“Dynamically route requests to regions with available capacity”
↩︎ Cross-region and cross-platform fallback, and feature parity“Are required for data residency and compliance requirements”
↩︎ Cross-region and cross-platform fallback, and feature parity“Bedrock limits request payloads to 20 MB.”
↩︎ Cross-region and cross-platform fallback, and feature parity - 6.
“Note that Anthropic model availability varies by region.”
↩︎ Cross-region and cross-platform fallback, and feature parity“Lifecycle dates on partner-operated platforms are set by the partner and can differ from the Claude API schedule.”
↩︎ Cross-region and cross-platform fallback, and feature parity