CertSafari
    Anthropic Claude Certified Architect – Professional (CCAR-P) - December 7 onwards· Lessons

    Domain 1 · Lesson 2/39

    End-to-end architecture evaluation: finding the component that fails first

    End-to-End Architecture Evaluation

    17 min read
    3% of exam
    5 sources
    Published 18 Nov 2026
    Docs as of 18 Nov 2026

    What you will be able to do

    • Trace a proposed Claude architecture through its input, processing, output and feedback stages, and say which limit governs each component
    • Identify the component that fails first under a stated load, using organization-level rate limits, ramp limits and spend caps
    • Decide whether an architecture enforces a stated access-control requirement or only asks the model to respect it
    • Judge whether a design meets a maintainability requirement for model and prompt changes, using pinned model IDs and regression gates

    Key concept

    First-failing component — An end-to-end architecture is a chain of components, and each one has its own limit: a quota, a credential scope, a lifecycle. To evaluate the architecture against a requirement, you find the component whose limit is reached first under the stated load or trust condition, because that component sets what the whole system can do.

    1.Tracing an architecture through input, processing, output and feedback

    Evaluating an architecture means following a request through every component and checking each stated requirement at each stage. A typical Claude system has four stages. Input is how work arrives, such as uploads, tickets or a queue. Processing covers the calls to Claude and the tools it uses. Output is where the results go, such as a database, a reply or a file. Feedback is how the team learns that something got worse and changes it. A requirement like "handle month-end volume" or "customers see only their own records" is not met by the architecture as a whole. It is met or broken at a specific stage, by a specific component.

    Each component is governed by its own limit, and one limit cannot stand in for another. The Managed Agents runtime makes this concrete. It splits an agent into parts that run in different places, and each part answers to a different control. The table below is a useful model for any review: one row per component, and the question "what actually limits it?" for each.

    Where each part of a Managed Agents deployment runs and what limits it
    PartWhere it runsWhat limits it
    Agent loopAnthropic's serversThe tool set, permission policies, and budget that you set
    bash and file toolsThe sandbox (an isolated Linux container per session in a cloud environment)Environment networking, the tool set, and permission policies
    Vault secretOutside the sandboxFor an MCP credential, the MCP server URL it is for
    Custom toolsYour applicationYour code

    The last row is the one reviewers most often skip. A hosted runtime's settings do not secure custom tools, MCP servers or self-hosted infrastructure. Whatever your own code does, your own code has to limit. Tightening a control on one row also leaves the other rows unchanged. Restricting which hosts an environment can reach does not stop an agent from using a credential on a host you did allow.

    Coupling also counts as a finding. In an early design, Anthropic ran the session, harness and sandbox in a single container. When that container failed, the session was lost. Moving the session log outside the harness meant a crashed harness could be replaced and the run resumed from the last event. When you review an architecture, ask which component's failure takes others down with it. That component usually deserves the first look.

    Checkpoint 1 of 7· Match them up

    You are reviewing a Managed Agents deployment. Match each part to the control that actually limits it.

    Tap a term, then the definition that fits it.

    Sources12

    2.Finding the first failure under load

    Under a load spike, the component that fails first is the one with a hard ceiling that cannot stretch. Elastic parts like autoscaling worker pools and durable queues are built to absorb a spike. A fixed per-minute quota is not. In a Claude pipeline, that fixed ceiling is usually the API rate limit.

    Rate limits belong to the organization, not to each worker. The Messages API measures requests per minute (RPM), input tokens per minute (ITPM) and output tokens per minute (OTPM), separately for each model class. You can set lower limits on individual workspaces, but you can never exceed what the tier allows. When ten times as many workers share one key, they compete for the same budget, so autoscaling the pool brings the system to the ceiling sooner. The queue still holds the backlog durably. The workers keep receiving 429s.

    How quickly the load arrives also matters. The API uses a token bucket, which refills capacity continuously, so a limit of 60 RPM can be enforced as one request per second. A short burst can therefore trip the limit even when the minute's total looks fine. Separately, a ramp limit caps how fast an organization's usage on a model can grow. A sudden tenfold rise can return 429s even while usage is below the ITPM and OTPM limits. These limits are also maximums, not guaranteed capacity.

    A month-end surge can also hit a second ceiling: the monthly spend cap. Each of the Start, Build and Scale tiers has one, at $500, $1,000 and $200,000. Once an organization reaches it, every request fails until the next month begins. The error looks like a rate limit but behaves differently, so the pipeline has to tell the two apart.

    The 429 returned when the tier's monthly spend cap is reached; the error_code separates it from an ordinary rate limitjson
    {
      "type": "error",
      "error": {
        "type": "rate_limit_error",
        "message": "You have reached your API usage limits: your organization has crossed its monthly API usage threshold, set based on your organization's API tier. You will regain access on 2026-09-01 at 00:00 UTC.",
        "details": { "error_code": "enforced_spend_limit_reached" }
      },
      "request_id": "req_018EeWyXxfu5pfWkrYcMdjWG"
    }
    Three limit responses a scaled-out pipeline can receive, and what each one means
    ConditionResponseDoes retrying help?
    RPM, ITPM or OTPM limit exceededHTTP 429, error names the limit, may carry retry-afterYes, after the retry-after interval
    Tier monthly spend cap reachedHTTP 429 rate_limit_error with error_code enforced_spend_limit_reached, no retry-afterNo, retries fail until access resumes or the tier is raised
    Your own spend limit reachedHTTP 400 invalid_request_errorNo, raise or remove the limit

    The cheapest way to raise the ceiling is often caching, not a higher tier. ITPM counts only uncached input, and tokens read from the cache do not count toward it. With a 2,000,000 ITPM limit and an 80 percent cache hit rate, a pipeline can process 10,000,000 input tokens per minute. Extraction pipelines that reuse one long system prompt for every document benefit the most. How to place cache breakpoints is covered in cache breakpoints and reusable packaging, and how to diagnose a binding constraint from metrics is in relieving the binding performance constraint.

    Checkpoint 2 of 7· Check yourself

    Midway through a month-end surge, every worker starts getting HTTP 429 rate_limit_error responses with error_code enforced_spend_limit_reached and no retry-after header. The SDK's automatic retries are on. What is happening, and what should the pipeline do?

    Checkpoint 3 of 7· Exam question

    An insurer runs a claims pipeline: uploads land in a managed durable queue, an autoscaling worker pool calls the Claude Messages API with one shared API key to extract fields, and results are written to a claims database. At month end, intake volume rises tenfold for several hours. Which component is the first to fail under this load?

    Sources3

    3.Checking where an access-control requirement is actually enforced

    An access-control requirement is met only by a component that can refuse a request. Instructions to the model cannot refuse anything. Claude is resilient to prompt injection, but model behavior is not a security control. A ticket or document containing hidden instructions can steer the agent, and the agent may also use any tool or credential it can reach to get past an obstacle. If the only thing keeping customer A out of customer B's orders is a sentence in the system prompt, the requirement depends on the model following it every time.

    Consider a support agent that calls a CRM tool server with one shared service-account credential, under the requirement that each customer sees only their own orders. Three common fixes do not meet that requirement.

    • Putting the customer ID in the system prompt and asking Claude to filter by it leaves enforcement with the model, and injected text can change the filter.
    • Running one agent per customer segment narrows the prompt, but each agent still holds a credential that can read every segment's records.
    • Scrubbing the final reply comes too late. An agent can include anything it is allowed to read in its reply or in tool output, and other customers' records were already read.

    The fix that works is to scope the credential to the person. Pass the signed-in user's identity through to the tool server, and have the server return only that user's records. The model then cannot see the forbidden data at all, whatever text it reads.

    The same reasoning decides which component fails first when a trust condition changes, for example when untrusted input starts reaching the agent. The first to fail is whichever reachable credential grants the most. Anthropic found this in its own early design. Untrusted code ran in the same container as the credentials, so a prompt injection only had to convince Claude to read its own environment. The fix was structural: tokens are kept in a vault or bundled into a resource during setup, and MCP calls go through a proxy that fetches credentials. The harness never holds them. Credential and identity models are covered in authentication and authorization posture, and injection controls in prompt injection surface and controls.

    Checkpoint 4 of 7· Check yourself

    An HR assistant built on Claude answers payroll questions. It calls a payroll tool server with one shared service-account credential that can read every employee's payslips. The requirement is that each employee sees only their own payslips. Which design change meets that requirement?

    Checkpoint 5 of 7· Exam question

    A retail company's support agent, built with the Claude Agent SDK, calls a CRM tool server using one shared service-account credential. The requirement states that a customer may only ever see their own order records. Which change makes the architecture satisfy this access-control requirement?

    Sources12

    4.Judging maintainability by how safely the system changes

    For a Claude system, a maintainability requirement usually means two things: you can change the model or the prompt without an outage, and you can undo a bad change quickly. An architecture meets it when every change is deliberate, applied in one place and checked before it reaches users.

    Model identifiers make this possible because they are pinned. A model ID always refers to the same model, and Anthropic does not update the weights or configuration of an existing ID. A newer version ships under a new ID. That makes a model swap an explicit configuration change you can review, roll out and roll back. Older Claude API aliases work differently. An alias such as claude-sonnet-4-5 points to the most recent dated snapshot, so a system calling it can change behavior without anyone deciding to change it. Pointing production at a moving alias trades a controlled change for an uncontrolled one. Even on a pinned ID, infrastructure updates can cause small differences in behavior, which is one more reason to keep measuring after release.

    Two changes together meet the requirement. First, move the prompts and model IDs into one shared, versioned configuration store that all six services read at runtime. A rollback then means repointing a single version, not redeploying six services that might drift apart. Moving each copy into a config file inside each service's own repo is still six copies with six deploy cycles. Second, put every prompt or model change through a regression suite of labeled examples, and promote it only when the scores hold. Regression detection is what keeps a prompt edit from quietly lowering accuracy. Watching production thumbs-down rates is a lagging signal: it reports the damage after users have already received it.

    The same principle explains why Managed Agents defines stable interfaces for the session, harness and sandbox. Harness assumptions go stale as models improve. Context resets added for one model became dead weight on the next. An architecture that keeps the changeable parts behind one interface and one configuration point can replace them without touching everything else. Managing model upgrades is covered in model version migration, and building the labeled examples is covered in deriving evaluations from business processes.

    Checkpoint 6 of 7· Check yourself

    A harness includes context resets that were added because an earlier model wrapped up tasks early as its context limit approached. On a newer model that behavior is gone, so the resets are now dead weight. Which design property lets the team drop them with the least risk to the rest of the system?

    Checkpoint 7 of 7· Exam question

    A team has a document-classification workflow spread across six services. Each service embeds its own copy of the prompt text and a hardcoded model identifier. Requirements: swap model versions without outages and roll back a bad prompt change within minutes. Choose two changes that best satisfy the maintainability requirement.(Select 2)

    0 of 2 selected

    Sources452

    Sources

    (5)

    Every claim above is drawn from one of these pages, quoted as it was written on the date shown.

    1. 1.
      “The settings do not secure your custom tools, your MCP servers, or your self-hosted infrastructure.”
      ↩︎ Tracing an architecture through input, processing, output and feedback
      “An agent can still put data it is allowed to read into its own reply or tool output.”
      ↩︎ Checking where an access-control requirement is actually enforced
      “Claude is inherently resilient to it, but model behavior is not a security control.”
      ↩︎ Exam trap 3
      “Agent loop | Anthropic’s servers | The tool set, permission policies, and budget that you set”
      ↩︎ Checkpoint
      “Limit a key’s permissions at the service that issued it, because these settings do not limit them.”
      ↩︎ Checkpoint
    2. 2.
      “a prompt injection only had to convince Claude to read its own environment.”
      ↩︎ Checking where an access-control requirement is actually enforced
      “The harness is never made aware of any credentials.”
      ↩︎ Checking where an access-control requirement is actually enforced
      “Each became an interface that made few assumptions about the others, and each could fail or be replaced independently.”
      ↩︎ Key concept
      “This allows the implementation of each to be swapped without disturbing the others.”
      ↩︎ Checkpoint
    3. 3.
      “Rate limits set the maximum number of API requests an organization can make over a defined period of time.”
      ↩︎ Finding the first failure under load
      “Short bursts of requests can exceed the limit and trigger rate limit errors.”
      ↩︎ Finding the first failure under load
      “A burst of requests within a few seconds can also count as a sharp increase.”
      ↩︎ Finding the first failure under load
      “With effective caching, you can substantially increase your actual throughput without raising your rate limits.”
      ↩︎ Finding the first failure under load
      “Limits are set at the organization level.”
      ↩︎ Exam trap 1
      “The error type is rate_limit_error, the same as for a rate limit, but the response has no retry-after header.”
      ↩︎ Exam trap 2
      “If you exceed any of the rate limits, you get a 429 error describing which rate limit was exceeded.”
      ↩︎ Prediction
      “the response has no retry-after header. Retrying, including the SDK’s automatic retries, fails until access resumes.”
      ↩︎ Checkpoint
    4. 4.
      “When an updated version is available, it ships under a new model ID.”
      ↩︎ Judging maintainability by how safely the system changes
      “An alias such as claude-sonnet-4-5 is a convenience pointer that resolves to the most recent dated snapshot for that minor version.”
      ↩︎ Judging maintainability by how safely the system changes
      “infrastructure updates produce minor differences in observable behavior even when the model ID and weights have not changed.”
      ↩︎ Judging maintainability by how safely the system changes
      “Anthropic does not update the weights or configuration of an existing model ID.”
      ↩︎ Exam trap 4
    5. 5.
      “Regression detection: Ensuring prompt changes don't accidentally hurt accuracy”
      ↩︎ Judging maintainability by how safely the system changes

    Ready to test yourself?

    Practise Anthropic Claude Certified Architect – Professional (CCAR-P) - December 7 onwards in quiz mode.

    Spotted a mistake, or was something unclear? Tell us.