Prompt caching (cache breakpoints)

Anthropic models can cache a stable prefix of a prompt so it does not have to be re-processed on every call. You mark the prefix with a cache breakpoint — cache_control on a message content part — and the provider writes it to its cache on the first call and reads it back on later calls. ultrarouter accepts that breakpoint on the OpenAI-compatible surface, carries it through unchanged, and translates it for the Anthropic adapter. This is how a large, stable system prefix — for example a distilled context module — is served as a cached prefix.

The shape you send

Put cache_control on the content part you want cached (not on the message). The supported value is { "type": "ephemeral" }, optionally with a ttl hint ("5m" — the default — or "1h"):

{
  "model": "anthropic/claude-haiku-4-5-20251001",
  "messages": [
    {
      "role": "system",
      "content": [
        {
          "type": "text",
          "text": "<a large, stable prefix — e.g. a reference guide or a learned module>",
          "cache_control": { "type": "ephemeral" }
        }
      ]
    },
    { "role": "user", "content": "<the volatile turn>" }
  ]
}

The breakpoint marks the boundary of the cacheable prefix: everything up to and including the annotated part is eligible to be cached. Put the stable content first (system prefix, long reference context, few-shot exemplars) and keep the volatile user turn after it.

What the gateway does

• Anthropic — the breakpoint is translated to cache_control on the corresponding Anthropic content block. A system part that carries a breakpoint is sent as an Anthropic system block (rather than a plain string) so the marker survives; a plain system prompt with no breakpoint is still sent as a plain string (unchanged behaviour).
• Every other provider — OpenAI-compatible providers do not accept cache_control, so the gateway strips it before dispatch and ignores it (recorded as a structured log, never an error). Your request still succeeds; it simply is not cached.

The provider must still meet its own minimum cacheable-prefix size (Anthropic requires a prefix on the order of a thousand tokens before it will cache it); a prefix below that threshold is sent normally and is not cached.

Cost accounting

Cost accounting reflects the provider-reported cache usage. Cache-read tokens are billed at the model's cache-read rate and cache-write (creation) tokens at the model's cache-write rate, with the remaining prompt tokens at the input rate; output tokens bill at the output rate. All amounts are integer micro-USD. The response usage.prompt_tokens_details.cached_tokens carries the provider-reported cache-read token count.

Notes

• Cache behaviour is provider-side and time-sensitive: a cache entry expires after its ttl, and whether a later call reads the cache depends on the provider. Treat the breakpoint as a hint, not a promise.
• The dated Anthropic snapshot ids (for example anthropic/claude-haiku-4-5-20251001) resolve their price from the curated logical model, so a call to a dated id is priced correctly rather than recorded as zero.