Managing Long Conversations: Compaction, Context Editing and Cache Trade-offs

1.Server-side compaction

Anthropic's context engineering guidance says to find the smallest possible set of high-signal tokens that make the result you want most likely. The model's attention is finite, so every token you add takes some of it.

When a conversation or agent task keeps growing, the main tool for this is server-side compaction. Claude writes a summary of the older turns on the server, and that summary replaces them. You write no summarization code. It keeps the task inside the window and keeps active context small. It is in beta for Claude 4.6 and later models and Claude Mythos Preview.

Pick on-demand when your application has to control when compaction happens, can't pause while a summary is written, or must keep recent turns word for word along with their thinking. Pick threshold compaction when you want the API to handle context inside ordinary requests. If the default summary drops something a later turn needs, you can supply your own summarization prompt.

Threshold compaction works differently from on-demand in a few ways. You set a trigger, and the API compacts when input tokens reach it. You add one parameter to your ordinary requests. You append the response as usual, and the API drops what came before the compaction block. It runs inside the request that reaches the threshold, so it cannot run in the background. You can keep recent turns word for word only by pausing after compaction and re-inserting them. On Claude Fable 5.1, Claude Opus 5.5, and Claude Sonnet 5.5, remove or drop the thinking in the turns you re-insert.

Compaction keeps you under the window, and the window has hard edges. If the input alone already exceeds the model's context window, the API returns a 400 invalid_request_error ("prompt is too long") on every model.

Input tokens plus max_tokens is a different case. On Claude 4.5 models and newer, if that sum exceeds the context window size, the API still accepts the request. If generation then reaches the window limit, it stops with stop_reason: "model_context_window_exceeded". On earlier models, the API returns a validation error instead. To opt in to the model_context_window_exceeded behavior on those models, use the model-context-window-exceeded-2025-08-26 beta header. To stay within the limits, estimate usage with the token counting API before you send.

Sources123

2.Context editing: clearing by rule

Compaction summarizes. Context editing removes content by rule instead, and it runs on the server before the prompt reaches Claude. It requires the context-management-2025-06-27 beta header. There are two strategies.

Tool result clearing (clear_tool_uses_20250919) is for agents that use tools heavily. Once Claude has processed an old file read or search result, it is rarely needed again. When the threshold is crossed, the API clears the oldest results first and puts placeholder text in their place. By default only results are cleared. Set clear_tool_inputs to true to clear the tool call parameters too.

    context_management={
        "edits": [
            {
                "type": "clear_tool_uses_20250919",
                # Trigger clearing when threshold is exceeded
                "trigger": {"type": "input_tokens", "value": 30000},
                # Number of tool uses to keep after clearing
                "keep": {"type": "tool_uses", "value": 3},
                # Optional: Clear at least this many tokens
                "clear_at_least": {"type": "input_tokens", "value": 5000},
                # Exclude these tools from being cleared
                "exclude_tools": ["web_search"],
            }
        ]
    },

Thinking block clearing (clear_thinking_20251015) sets how many recent assistant turns keep their thinking. The default depends on the model: all turns on Opus 4.5+, Sonnet 4.6+ and the Fable and Mythos models, and only the last turn on earlier Opus and Sonnet models and all Haiku models.

Two more details. First, your client keeps the full, unedited history and never needs to sync with the edited version. Second, if you pair context editing with the memory tool, Claude gets an automatic warning as context nears the clearing threshold, so it can save important information to memory files before it is cleared.

    context_management={
        "edits": [
            {
                "type": "clear_thinking_20251015",
                "keep": {"type": " ? ", "value": 2},
            }
        ]
    },

Sources4

3.Keeping tool context small

Clearing by rule is not the only way to control what tools cost you in context. Everything in the request counts toward the window, including your tool definitions. To reduce the context the definitions themselves consume, see Manage tool context, or defer tool definitions with the tool search tool.

Another lever is programmatic tool calling, which Anthropic's cookbook lists among the ways to keep large results out of the window entirely. Prompt caching helps with the cost of repeated prefixes, but it changes what you pay for those tokens, not whether they count toward the window.

Sources15

4.Context management versus the prompt cache

Prompt caching and context management solve different problems. Cached prefixes still sit in the context window. Caching changes what you pay for those tokens, not whether they count. So caching never rescues a conversation that is about to hit the limit.

The two do interact. Tool result clearing invalidates cached prompt prefixes whenever it clears something, and each clearing triggers a new cache write. That is the reason for clear_at_least: if the API can't free at least that many tokens, it skips the clearing, so you only break the cache when the savings justify it. Thinking block clearing works the same way. Kept thinking preserves the cache, and clearing invalidates it at the point where clearing happens. Set keep according to which matters more to you, cache performance or free context space.

There is one model-specific nuance. On Claude Fable 5.1, Claude Opus 5.5, and Claude Sonnet 5.5, server-side context management never invalidates thinking blocks. Client-side edits to earlier turns can invalidate the thinking blocks in every later assistant turn.

To preview a configuration before you commit to it, pass the same context_management block to client.beta.messages.count_tokens. The response returns both the count after clearing and the original count.

{
  "input_tokens": 25000,
  "context_management": {
    "original_input_tokens": 70000
  }
}

Sources4

5.Client-side SDK compaction and when it fits

There is also a client-side option. The TypeScript and Ruby SDKs offer compaction when you use tool_runner, turned on by adding compaction_control. The docs say server-side compaction is generally preferred. The SDK version watches token usage after each response. Once usage passes context_token_threshold (default 100,000), it asks for a summary in <summary></summary> tags and replaces the entire history with it. You can name a cheaper model to write summaries, or pass a custom summary_prompt. A lower threshold compacts more often and keeps contexts smaller. A higher one keeps more context but risks hitting the limit.

One known problem: the SDK adds input, cache-creation, cache-read and output tokens together. After server-side tools such as web search, cache_read_input_tokens includes reads from the tool's internal calls. The total then looks inflated and compaction fires too early. The workarounds are to get an accurate length from the token counting endpoint, or to avoid this compaction when you rely heavily on server tools. It works best for long tasks with measurable progress whose artifacts persist outside the conversation. It is a poor fit when a task needs exact recall of early details or has to track exact state across many variables.

Sources4

Exam traps

Each one states something that sounds right. Open it to see what is actually true.

1. 1.Prompt caching frees up context window space for long conversations.Why is that wrong?

   Cached tokens still count toward the window. Caching only lowers what you pay for them.

   Covered in Context management versus the prompt cache

2. 2.After server-side context editing clears tool results, your client has to rewrite its local history to match.Why is that wrong?

   Editing happens on the server before Claude sees the prompt. The client keeps sending its full history.

   Covered in Context editing: clearing by rule

3. 3.The SDK's summed usage is always an accurate measure of conversation length.Why is that wrong?

   After server-side tools, cache_read_input_tokens includes the tool's internal reads, so the SDK can trigger compaction too early.

   Covered in Client-side SDK compaction and when it fits

Practise it for real

Measure how many tokens a tool result clearing configuration would save before you enable it in production

1. 1.Take a long agent conversation with many tool results and run client.messages.count_tokens on it with no context management

   Why: You need a baseline input count

   You should see: A response with a single input_tokens value

2. 2.Run client.beta.messages.count_tokens again with betas=["context-management-2025-06-27"] and a clear_tool_uses_20250919 edit with a trigger below your baseline

   Why: The counting endpoint can preview the effect of context editing

   You should see: input_tokens comes back lower than context_management.original_input_tokens

3. 3.Add clear_at_least and raise it until clearing stops applying

   Why: Clearing invalidates the prompt cache, so find the point where the savings stop being worth the cache rewrite

   You should see: Once the API can't clear at least that amount, the strategy is not applied