CertSafari
    Anthropic Claude Certified Architect – Professional (CCAR-P) - December 7 onwards· Lessons

    Domain 3 · Lesson 20/39

    Production reliability controls: rate limits, retries, idempotency and fallback

    Production Reliability Controls

    18 min read
    2.63% of exam
    7 sources
    Published 18 Nov 2026
    Docs as of 18 Nov 2026

    What you will be able to do

    • Read a 429 or 5xx trace and tell a rate limit, a ramp limit and a spend cap apart by status code, error_code and the retry-after header
    • Pick the retry and backoff behaviour that fits each transient failure, and recognise failures where retrying can never succeed
    • Make a side-effecting tool safe to retry with an idempotency key, so a timeout does not produce a duplicate effect
    • Route refused or capacity-constrained requests to a fallback, and configure that fallback on every request path

    Key concept

    Classify the failure before you retry it — Every reliability control depends on first reading what kind of failure happened. The status code, the error_code and whether a retry-after header is present tell you if waiting helps, if pacing helps, if a different model helps, or if nothing automatic will help.

    1.Reading the failure before choosing a control

    A production integration on the Claude API fails in a small number of recognisable ways, and each one needs a different control. Retrying is the right answer for some and the wrong answer for others. If you retry a failure that cannot clear, the queue grows. If you don't retry one that would have cleared, work is lost for nothing. The first job in any incident is to read the trace closely enough to tell which failure you have.

    HTTP 429 alone does not settle it. The rate_limit_error type covers three conditions: a per-minute rate limit, a ramp limit after a sharp rise in traffic, and the monthly spend cap of your usage tier. What separates them is error.details.error_code and whether the response carries a retry-after header. Spend limits you set yourself behave differently again: they come back as HTTP 400 invalid_request_error, and the message says when access resumes.

    Failure signals in a Claude API trace and the control each one calls for
    Signal in the traceWhat happenedControl
    429 rate_limit_error with retry-afterA per-minute RPM, ITPM or OTPM limit was exceededWait the retry-after seconds, then pace traffic
    429 with error_code ramp_limit_exceededUsage grew faster than the ramp limit allowsHonour retry-after and ramp concurrency up gradually
    429 with error_code enforced_spend_limit_reached, no retry-afterThe tier's monthly spend cap was reachedStop retrying, park the work, alert someone to raise the limit or move tiers
    400 invalid_request_error 'You have reached your specified API usage limits'A spend limit you set yourself was reachedRaise or remove the limit; retrying will not help
    500 api_error or 529 overloaded_errorTransient server-side failure or capacity pressureRetry with exponential backoff
    200 with stop_reason refusalA safety classifier declined the requestRetry on a fallback model

    A spend-cap 429 is not transient. When an organization reaches its tier's monthly cap, API usage pauses until 00:00 UTC on the first day of the next month. The error type is the same as for a rate limit, but the response has no retry-after header. The default SDK retries these anyway and fails every time. Limits are set per organization, so moving traffic to another model class draws on the same exhausted budget. Lowering concurrency to fit under RPM doesn't help either, because RPM was never the constraint. The control that works is to branch on the error_code, stop retrying, hold the queued work, and alert whoever can raise the limit or move to a higher tier. One exception matters if your traces include Claude Code: limits on the Claude Code workspace are checked separately, and requests over that limit can get a 429 that does carry retry-after.

    Checkpoint 1 of 8· Match them up

    Match each signal in a production trace to the reliability control it calls for.

    Tap a term, then the definition that fits it.

    Checkpoint 2 of 8· Exam question

    Your team runs an ingestion service on the Messages API through the official SDK with default settings. For several hours every request fails with HTTP 429 rate_limit_error. The responses carry no retry-after header, and the body includes error.details.error_code set to enforced_spend_limit_reached. Worker pools keep retrying and the queue depth grows. Which reliability control addresses this failure pattern?

    Sources12

    2.Pacing traffic under rate limits and ramp limits

    A true rate limit clears if you wait. The Messages API measures requests per minute (RPM), input tokens per minute (ITPM) and output tokens per minute (OTPM) for each model class. A 429 response says which of these was exceeded. Capacity refills continuously under a token bucket algorithm instead of resetting each minute. Limits can also be enforced over shorter windows: a 60 RPM limit might be applied as one request per second, so a burst can trip it even when the minute's total looks fine. When the response has a retry-after header, wait that many seconds before sending again.

    Not every number in the request affects these limits. Only uncached input tokens and cache writes count toward ITPM. Cache reads don't, which is why caching raises effective throughput (prompt caching and cache breakpoints). OTPM counts the output tokens actually generated, and max_tokens plays no part in it. Lowering max_tokens is therefore never a fix for a rate-limit 429. Deciding which limit is your binding constraint in the first place is covered in relieving the binding performance constraint.

    A ramp limit caps how fast usage can grow, not how high it can go. Your ramp limit is your organization's highest recent per-minute usage of a model plus a step. Each minute you can go up to one step above that peak, until you reach your rate limit. After a quiet period the ramp limit drops back toward a starting amount. A nightly job that starts 400 workers at once from idle is exactly the case where this bites: requests return 429 with error_code ramp_limit_exceeded while the ITPM and OTPM dashboards show plenty of headroom. A burst of requests within a few seconds also counts as a sharp increase.

    Two controls address a ramp-limit 429, and they work together. Each failed call waits at least the retry-after seconds before it is sent again. The job as a whole starts at a request rate that succeeds and adds load gradually, spacing calls evenly, so the ramp limit rises one step per minute alongside it. Requesting a higher ITPM tier does nothing here, because the job was never near ITPM. A fixed short delay such as 100 ms ignores retry-after and keeps the burst going. Work that doesn't need an immediate answer can go through the Message Batches API instead, which ramp limits don't apply to.

    Checkpoint 3 of 8· Check yourself

    A nightly job sits idle all day, then starts 400 workers at once. Within seconds many calls return HTTP 429 with error_code ramp_limit_exceeded, but the ITPM and OTPM dashboards show plenty of headroom. Which change fixes it?

    Checkpoint 4 of 8· Exam question

    A nightly batch re-scoring job on the Messages API starts at 02:00 by launching 400 concurrent workers from an idle state. Within seconds many calls return 429 with error.details.error_code of ramp_limit_exceeded, even though the ITPM and OTPM dashboards sit well below the configured limits. The responses include retry-after. Choose two controls that address this pattern.(Select 2)

    0 of 2 selected

    Sources2

    3.Retry and backoff for transient errors

    Server errors and overload are the failures backoff is for. A 500 api_error means something went wrong inside Anthropic's systems. Retry it with exponential backoff, and if it persists, contact support with the request ID. A 529 overloaded_error means the API is temporarily overloaded. The official SDK already retries connection errors, rate limits and 5xx errors with exponential backoff, twice by default, and honours retry-after when the header is present. Use max_retries to raise, lower or disable this.

    Some errors should never be retried as they are. A 400 invalid_request_error means the format or content of the request is wrong, and sending the same request again fails the same way. Timeouts are a different trap. A response that takes longer to generate than the client's timeout will time out on every re-send, so retrying a non-streaming request that is too slow just repeats the wait. Long-running requests belong on the streaming Messages API, or on the Message Batches API, where you poll for results instead of holding one network connection open.

    Claude Code has its own retry settings for unattended runs. CLAUDE_CODE_MAX_RETRIES defaults to 10. Setting CLAUDE_CODE_RETRY_WATCHDOG=1 in a CI job retries 429 and 529 capacity errors indefinitely. Even with the watchdog set, a standard-speed request fails at once when it gets a 429 reporting a spend limit or exhausted usage credits. CLAUDE_CODE_NONSTREAMING_TIMEOUT_RETRIES limits how many times a timed-out non-streaming request is sent again. A low value makes a request that will never fit in the timeout fail sooner.

    Checkpoint 5 of 8· Check yourself

    Your report-generation service sends large non-streaming Messages API requests. Every few minutes one times out on the client, and the SDK's retries time out again on each re-send. Nothing in the trace shows a 429, 500 or 529. What change stops these failures?

    Sources13

    4.Idempotency for side-effecting tools

    A timeout tells you that you got no answer. It does not tell you the work wasn't done. Say a support agent's issue_refund tool calls a payments service, the service completes the refund, and the wrapper's HTTP call times out before reading the response. A wrapper that retries now issues a second refund. Backoff makes this worse in a different way: a longer pause before the retry still produces the duplicate, it just arrives later. Removing the retry stops the duplicate, but it turns every transient blip into a failed refund the user has to submit again.

    The control that keeps the retry and removes the duplicate is an idempotency key the receiving service remembers. The wrapper builds one stable key per intended effect and sends it on the first attempt and on every retry. One way is to combine the business identifier (the order id) with the tool_use_id of the call Claude made. When a retry arrives with a key the service has already seen, it returns the stored result instead of running the refund again.

    A robust payments service goes further and stores the key as a record with its own lifecycle: it claims the key in the same transaction that creates the pending refund, executes the refund, then finalizes the record with the outcome. A bare header check, which looks the key up, runs the refund if the key is absent and records the key afterwards, leaves a gap. If the service completes the refund but dies before recording it, the retry finds no key and refunds a second time. Claiming the key first closes that gap.

    The service should also offer a lookup by key, for example GET /refunds?idempotency_key=.... It serves the client that lost the response to a request that actually succeeded: that client reads the recorded outcome instead of guessing by retrying a write.

    Asking Claude in the system prompt to check refund history first is not a substitute. The duplicate comes from the wrapper retrying, not from Claude deciding to call the tool twice.

    When a tool call does fail, report that to Claude instead of hiding it. Return the error in a tool_result with is_error set to true, and Claude will work the failure into its reply. A service built around unique keys may answer a conflicting replay with 409 conflict_error. The wrapper should treat that as "already done", not as a new failure to retry.

    Returning a tool execution failure to Claude as an error resultjson
    {
      "role": "user",
      "content": [
        {
          "type": "tool_result",
          "tool_use_id": "toolu_01A09q90qw90lq917835lq9",
          "content": "ConnectionError: the weather service API is not available (HTTP 500)",
          "is_error": true
        }
      ]
    }

    Checkpoint 6 of 8· Fill the gap

    Your issue_refund wrapper has given up on a payments call that kept failing. Rather than hiding the failure, you want to report it to Claude. Which field name completes this tool_result so Claude treats the content as a failed tool execution and explains it in its reply?

    {
      "role": "user",
      "content": [
        {
          "type": "tool_result",
          "tool_use_id": "toolu_01A09q90qw90lq917835lq9",
          "content": "ConnectionError: the weather service API is not available (HTTP 500)",
          " ? ": true
        }
      ]
    }

    Checkpoint 7 of 8· Exam question

    A support agent built on the Messages API calls an issue_refund tool that wraps an internal payments service. Traces show the payments service completed the refund, but the tool wrapper's HTTP call timed out before reading the response and retried, so roughly one in fifty timeouts produced a double refund. Which control removes the duplicate effect while keeping the retry?

    Sources145

    5.Fallback routing for refusals and capacity

    They don't show up at all, because a refusal is a successful response. Claude Fable, Opus 5.5, Opus 5, Sonnet 5.5 and Haiku 5.5 models include safety classifiers that can decline a request. The result is HTTP 200 with stop_reason: "refusal" and a stop_details object, not an error. Monitoring built on error rates misses it entirely, so emit one event per refusal and one per fallback-served response, and alert on the gap between the two counts (instrumentation and monitoring gaps). Branch on stop_reason, not on the content or the category field, which can be null. If a refusal arrives mid-stream, discard any partial output.

    Retrying a refusal on the same model usually gets another refusal, so the control here is fallback routing to a different model. With server-side fallback, in beta on the Claude API, you set fallbacks to "default" and the API retries on the model Anthropic recommends for that refusal category, all within one call. The SDK middleware does the same on the client side for any platform. It walks your fallback list in order and keeps later turns on the model that accepted. Claude Haiku 5.5 has no server-side fallback, and batches don't support it, so those paths need the middleware or a manual retry.

    Server-side fallback: one request, retried on a recommended model if the primary refusespython
    client = Anthropic()
    
    response = client.beta.messages.create(
        model="claude-fable-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": "Hello, Claude"}],
        fallbacks="default",
        betas=["server-side-fallback-2026-07-01"],
    )
    print(response.model)

    Fallback has to be set on each request. A retry handler, an error-recovery branch or a background worker that sends a request again without fallbacks loses the protection on exactly the requests most likely to need it. Subagent calls need their own fallback too, because the parameter doesn't carry over to model calls made from inside tool execution. Capacity has a similar routing option: with service_tier="auto", a request uses Priority Tier capacity when it's available and falls back to your other capacity when it isn't. Check usage.service_tier in the response to see which tier served it.

    Checkpoint 8 of 8· Fill the gap

    A support assistant on Claude Fable sometimes gets HTTP 200 responses with stop_reason "refusal". You want the API to retry each declined request on the model Anthropic recommends, all within the same call. Which parameter completes this request?

    client = Anthropic()
    
    response = client.beta.messages.create(
        model="claude-fable-5",
        max_tokens=1024,
        messages=[{"role": "user", "content": "Hello, Claude"}],
         ? ="default",
        betas=["server-side-fallback-2026-07-01"],
    )
    print(response.model)

    Sources67

    Sources

    (7)

    Every claim above is drawn from one of these pages, quoted as it was written on the date shown.

    1. 1.
      “Your organization has hit a rate limit, reached its usage tier’s monthly spend cap, or reached a spend limit on the Claude Code workspace.”
      ↩︎ Reading the failure before choosing a control
      “The official SDK automatically retries transient failures (such as connection errors, rate limits, and 5xx server errors) with exponential backoff, twice by default”
      ↩︎ Retry and backoff for transient errors
      “Retry the request with exponential backoff; if the error persists, contact support with the request ID.”
      ↩︎ Retry and backoff for transient errors
      “For long-running Messages API requests, use the streaming Messages API.”
      ↩︎ Retry and backoff for transient errors
      “For example, the resource was modified concurrently, or a value that must be unique is already in use.”
      ↩︎ Idempotency for side-effecting tools
      “A sharp increase in usage can also cause a 429 even while you’re under your rate limits”
      ↩︎ Checkpoint
    2. 2.
      “When usage reaches a spend limit you set, requests return HTTP 400 with error type invalid_request_error.”
      ↩︎ Reading the failure before choosing a control
      “Retrying, including the SDK’s automatic retries, fails until access resumes.”
      ↩︎ Reading the failure before choosing a control
      “Limits are set at the organization level.”
      ↩︎ Reading the failure before choosing a control
      “Claude Code requests over that workspace’s limit can instead receive a 429 that carries a retry-after header.”
      ↩︎ Reading the failure before choosing a control
      “If you exceed any of the rate limits, you get a 429 error describing which rate limit was exceeded.”
      ↩︎ Pacing traffic under rate limits and ramp limits
      “a rate of 60 requests per minute (RPM) might be enforced as 1 request per second”
      ↩︎ Pacing traffic under rate limits and ramp limits
      “The max_tokens parameter does not factor into OTPM rate limit calculations”
      ↩︎ Pacing traffic under rate limits and ramp limits
      “Each minute you can go up to a step above that peak, so the limit rises as your successful usage grows.”
      ↩︎ Pacing traffic under rate limits and ramp limits
      “On the Messages API, error.details.error_code is ramp_limit_exceeded.”
      ↩︎ Pacing traffic under rate limits and ramp limits
      “Wait at least the number of seconds in the retry-after header before retrying.”
      ↩︎ Pacing traffic under rate limits and ramp limits
      “The Message Batches API isn’t subject to ramp limits.”
      ↩︎ Pacing traffic under rate limits and ramp limits
      “On the Messages API, error.details.error_code is enforced_spend_limit_reached. Use it to tell this response apart from a rate limit.”
      ↩︎ Key concept
      “The error type is rate_limit_error, the same as for a rate limit, but the response has no retry-after header.”
      ↩︎ Exam trap 1
      “After a sharp increase in usage, requests can return a 429 error even while you’re under your ITPM and OTPM rate limits.”
      ↩︎ Exam trap 2
      “the same as for a rate limit, but the response has no retry-after header. Retrying, including the SDK’s automatic retries, fails until access resumes.”
      ↩︎ Checkpoint
    3. 3.
      “A response from Claude that takes longer than the timeout to generate times out again on every re-send”
      ↩︎ Retry and backoff for transient errors
      “Set to 1 in unattended sessions such as CI jobs to retry 429 and 529 capacity errors indefinitely”
      ↩︎ Retry and backoff for transient errors
    4. 4.
      “Claude will then incorporate this error into its response to the user.”
      ↩︎ Idempotency for side-effecting tools
    5. 5.
      “Idempotency became a stored resource with a claim→execute→finalize lifecycle”
      ↩︎ Idempotency for side-effecting tools
      “This closes the "processor succeeded but our server died before recording it" window”
      ↩︎ Idempotency for side-effecting tools
      “for clients that lose the response to a request that actually succeeded”
      ↩︎ Exam trap 3
    6. 6.
      “A refusal is an HTTP 200, so monitoring built on error rates or 5xx responses never sees it.”
      ↩︎ Fallback routing for refusals and capacity
      “A handler that re-issues a request without fallback loses the protection on exactly the requests most likely to need it.”
      ↩︎ Fallback routing for refusals and capacity
      “The fallbacks parameter does not carry over to model calls made from inside tool execution.”
      ↩︎ Fallback routing for refusals and capacity
      “Server-side fallback is not available for batches”
      ↩︎ Fallback routing for refusals and capacity
      “Re-sending a refused request to the same model usually earns another refusal.”
      ↩︎ Exam trap 4
    7. 7.
      “Uses the Priority Tier capacity if available, falling back to your other capacity if not”
      ↩︎ Fallback routing for refusals and capacity

    Ready to test yourself?

    Practise Anthropic Claude Certified Architect – Professional (CCAR-P) - December 7 onwards in quiz mode.

    Spotted a mistake, or was something unclear? Tell us.