Native passthrough (OpenAI Responses / Anthropic Messages)

ultrarouter speaks the OpenAI Responses API (POST /v1/responses) and the Anthropic Messages API (POST /v1/messages) natively. You point the official SDK at ultrarouter by changing only the base URL and the key — your request body is forwarded to the provider verbatim and the response is returned untouched. There is no routing, no compression, no model remapping and no field injection: the model your body names is the model that is called. The gateway only authenticates your ur_sk_* key, enforces your rate and spend limits, meters the call from the provider's own usage, tags it, and records a ledger row.

Use these two surfaces when you want the full native provider surface (the Responses hosted web_search tool, strict text.format JSON schema, reasoning, prompt_cache_key, store:false; the Messages output_config, cache_control, tool results and continuation blocks) to pass through unchanged. For the normalized, multi-provider surface, use POST /v1/chat/completions instead.

Set your base URL to your gateway (the invite-only hosted https://api.ultrarouter.io, the local stack at http://localhost:8080, or your own deployment) and your key to a ur_sk_* key with the gateway:invoke scope.

OpenAI Responses — POST /v1/responses

The OpenAI SDK sends the key as Authorization: Bearer. Change only baseURL + apiKey:

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'http://localhost:8080/v1',
  apiKey: process.env.UR_API_KEY, // your ur_sk_* key, NOT an OpenAI key
});

const res = await client.responses.create({
  model: 'gpt-6-sol',
  input: 'Draft a product update in two sentences.',
});

Over raw HTTP:

curl -sS -X POST "http://localhost:8080/v1/responses" \
  -H "Authorization: Bearer $UR_API_KEY" \
  -H "Content-Type: application/json" \
  -H 'X-UR-Call-Type: studio-writer' \
  -H 'X-UR-Trace: {"run":"run-42","case":"post-1","stage":"writer","attempt":1}' \
  -d '{ "model": "gpt-6-sol", "input": "Draft a product update in two sentences." }'

The response body is exactly OpenAI's. Usage is metered from usage (input_tokens, output_tokens, input_tokens_details.cached_tokens, output_tokens_details.reasoning_tokens — reasoning tokens are billed as output) plus a per-call fee for each hosted web_search_call output item.

Anthropic Messages — POST /v1/messages

The Anthropic SDK sends the key as the x-api-key header with an anthropic-version. POST /v1/messages accepts your ur_sk_* key via x-api-key or Authorization: Bearer:

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic({
  baseURL: 'http://localhost:8080',
  apiKey: process.env.UR_API_KEY, // sent as x-api-key; your ur_sk_* key, NOT an Anthropic key
});

const msg = await client.messages.create({
  model: 'claude-opus-5-5',
  max_tokens: 1024,
  messages: [{ role: 'user', content: 'Review this draft for tone.' }],
});

Over raw HTTP:

curl -sS -X POST "http://localhost:8080/v1/messages" \
  -H "x-api-key: $UR_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{ "model": "claude-opus-5-5", "max_tokens": 1024, "messages": [{ "role": "user", "content": "Review this draft for tone." }] }'

Usage is metered from usage.input_tokens + cache_read_input_tokens + cache_creation_input_tokens

• output_tokens, each cache bucket billed at its own rate.

The ur request headers

These extensions ride request headers (a native body cannot carry a ur block):

The response carries X-UR-Request-Id (the req_* id) on every call and X-UR-Cost-Micro-USD (the booked cost, integer micro-USD) on non-streaming calls.

Streaming

Send stream: true and the upstream SSE bytes are forwarded verbatim — never re-serialised. The gateway tee-reads the stream to meter usage (OpenAI's response.completed event, Anthropic's message_start + message_delta) without changing a byte. Because usage arrives at the end of the stream, X-UR-Cost-Micro-USD is returned only for non-streaming calls. For a streamed call, read the booked cost afterwards with GET /v1/generation?id=req_…, using the X-UR-Request-Id the response carried.

Reading the booked cost back

GET /v1/generation?id=req_… returns the call's booked cost (cost.total_micro_usd) and its served model. Two things decide what the lookup finds:

• The record lands just after the response ends. It is written off the response path, so a lookup sent the instant a stream finishes can return 404 for a moment. Retry for a few seconds before you treat the cost as missing.
• A request the gateway refused never has a record. A spend cap (402 spend_cap_exceeded), a rate limit (429 rate_limited), a bad key, an invalid body or an unpriced model (403 model_unpriced) is refused before any provider is called, in ultrarouter's own error envelope. Nothing was sent and nothing was booked against your caps, so its X-UR-Request-Id stays 404 not_found: record it as $0 and don't keep looking it up.

A call that did reach a provider always has a record, including an upstream error (status: "error") and a stream your client hung up on (status: "cancelled").

Errors and retries

An upstream error passes through with the provider's own status and body, and the upstream retry-after, retry-after-ms, x-should-retry and request-id / x-request-id headers are forwarded so your own client can decide what to do. The passthrough never retries an upstream call itself. Gateway-side failures — a missing key (authentication), a spend cap (spend_cap_exceeded), a deadline (provider_timeout) — use ultrarouter's normalized error envelope.

Each door forwards only to its own provider: /v1/responses to OpenAI and /v1/messages to Anthropic (or to a custom provider you registered). A model id that belongs to another provider — for example openai/gpt-5-mini on /v1/messages — is refused before dispatch with invalid_request (HTTP 400), so a provider key is never sent to the wrong API. Use POST /v1/chat/completions to reach any provider, or a passthrough rule with serve.protocol: "openai_chat" to serve a native request from an OpenAI-chat model.

Unpriced models

The gateway meters every call from the catalog price of the model your body names. If the requested model has no usable price — a model id the catalog has never priced, or a custom provider registered without pricing — the gateway cannot compute the call's cost. By default it refuses the call before dispatch with the normalized error model_unpriced (HTTP 403), exactly like /v1/chat/completions and /v1/embeddings:

{
  "error": {
    "code": "model_unpriced",
    "message": "model 'your-model-id' has no recorded price, so the gateway cannot meter a call to it and refuses it before dispatch. Use a priced model, declare `pricing` on the custom provider (0 is allowed), or ask for the model to be priced in the catalog.",
    "request_id": "req_…",
    "retryable": false
  }
}

This is deliberate: a call the gateway cannot cost would be recorded at $0, and a $0 that is really an unknown cost is invisible to your per-key and per-workspace spend caps — so an unpriced model id could run past a cap without limit. Refusing it keeps every served call visible to your caps and cost reporting. To use the model, price it: declare pricing on your custom provider (an explicit 0 is a valid, stated price), or ask for the model to be priced in the catalog.

A self-hosted deployment that wants the old behaviour — forward unpriced models and meter them at $0 with cost_basis: "unpriced" — can set UR_PASSTHROUGH_ALLOW_UNPRICED=true on the gateway. Only do this for a workload whose spend is bounded by something other than ultrarouter's caps, because those $0 rows do not count against them. The booked cost_basis is visible on GET /v1/generation?id=req_… under cost.cost_basis.

Stage routing and caching

A per-workspace passthrough routing policy lets the gateway swap the model of a POST /v1/responses or POST /v1/messages request from its call tags — so you can move a cheap stage of a pipeline (for example a "repair" pass) off your frontier model without changing a line of your code. The gateway reads the policy from the same 5-second versioned-config cache as the rest of your settings, so a change takes effect within one window.

Set it with PUT /v1/workspaces/{id}/passthrough-policy (owner/admin; audited; bumps your config version). Read it back with GET. The shape:

{
  "enabled": true,
  "rules": [
    {
      "id": "repair-lines-haiku",
      "match": { "call_type": "repair_lines", "trace_stage": "repair" },
      "serve": { "model": "claude-haiku-5-5" },
      "cache": "anthropic_prefix"
    }
  ]
}

A rule matches a request when every field present in match equals the request's X-UR-Call-Type, the stage in its X-UR-Trace, and/or the body's model. The first matching rule wins. Set enabled: false to turn the whole policy off (the kill switch) with no code change.

By default (serve.protocol: "native", the default) a rule serves the same provider family as the route (a native cross-family rewrite — matching an OpenAI model but serving an Anthropic one — is refused when you save the policy, naming the offending rule). When a native rule matches:

• The body's model is rewritten to serve.model. The served model must be priced and active in the catalog. If the served model has no usable price, the call is refused with model_unpriced exactly as above.
• If the served model does not accept temperature (a deprecated parameter on some models), it is dropped from the body — along with a paired top_p — and recorded so a determinism claim is never silently false.
• With cache: "anthropic_prefix" on a POST /v1/messages call, the gateway injects cache_control: { "type": "ephemeral" } on the system block (a string system becomes the one-block form) and on the largest text block of the first user message, idempotently (it never duplicates a cache_control you already set, and never touches any other field). For OpenAI POST /v1/responses only the model rewrite applies.

Everything else in your native request body is forwarded verbatim.

Serving another family's model (cross-family)

Set serve.protocol: "openai_chat" to serve a model that speaks OpenAI chat completions — any OpenAI-compatible open model (for example a DeepInfra model), or your own OpenAI-compatible custom provider — from either native door. The gateway translates the request into an OpenAI chat request, dispatches it through the same provider path as POST /v1/chat/completions (your BYOK or platform key, the SSRF egress guard for a custom endpoint), and translates the response and the stream back into the native API's exact shapes — so an app built on the official @anthropic-ai/sdk or openai SDK is served by an open model with no code change.

{
  "enabled": true,
  "rules": [
    {
      "id": "writer-to-open-model",
      "match": { "call_type": "short_write", "model": "claude-opus-5-5" },
      "serve": { "model": "deepinfra/glm-5.3", "protocol": "openai_chat" }
    }
  ]
}

• The served model must be chat-capable, active and priced; otherwise the rule is refused when you save the policy, naming it. Spend caps reserve and settle at the served model's price.
• The translation maps your native fields: an Anthropic system/text blocks, output_config.format (json_schema), adaptive/enabled thinking + output_config.effort, max_tokens, temperature, top_p, stop_sequences, and plain function tools/tool_choice; an OpenAI Responses instructions, input, text.format, reasoning.effort, max_output_tokens. A json_schema becomes a strict response format when the open model supports it, otherwise the gateway pins the schema in the system text and asks for a JSON object. The open model's reasoning text is dropped (never surfaced as a forged thinking block).
• Add serve.reasoning_effort ("low" | "medium" | "high") to force the reasoning effort on the translated request, overriding whatever the client asked for. Some open reasoning models, handed a high effort and a large max_tokens, spend the whole token budget reasoning and never emit an answer; serving them at a lower effort avoids that. It is valid only with serve.protocol: "openai_chat" (the policy is refused, naming the rule, otherwise). The request row records the client's requested_effort and the served_effort under routing_decision.passthrough_rule.

{
  "enabled": true,
  "rules": [
    {
      "id": "writer-low-effort",
      "match": { "call_type": "short_write" },
      "serve": { "model": "deepinfra/glm-5.3", "protocol": "openai_chat", "reasoning_effort": "medium" }
    }
  ]
}

• A request carrying a feature the open model cannot run — a server tool (web_search, …), an image or document block to a text-only model, a container, or citations — is served unchanged on its own family and the reason is recorded on the request row (routing_decision.translation_refused); it is never silently dropped and never an error to your client.
• The response model field is the model you asked for (so your SDK never chokes); the model actually served rides X-UR-Served-Model and model_served.
• On a streamed Anthropic-door call, message_start reports input_tokens: 0 because the open model counts the prompt only at the end of its stream; the final message_delta carries the real input_tokens (as Anthropic's own stream does). Read usage after the stream ends, as the Anthropic SDK's final message does.
• A reasoning open model can think for minutes before its first token. While it sends nothing, the gateway keeps the stream alive every 15 seconds the way the door itself does: an Anthropic ping event on /v1/messages, an SSE comment (: ping) on /v1/responses. Your SDK skips both. The per-request deadline still applies (300 s by default, up to 600 s with X-UR-Timeout-Ms); a slow model is better pinned to a lower serve.reasoning_effort.
• If your client hangs up mid-stream, the gateway stops the upstream call and records the call as cancelled.

Injecting a learned pack (inject)

A rule's action can also carry inject: { "experiment_pack_id": "xpk_…" } — alongside or instead of serve — to insert an owner-loaded learned pack into the matched request's learned reference slot, so the served model receives a short playbook as the first text of its first user message. An inject rule must match a call type (match.call_type; and should match an api_key_id so one key is the injected arm).

It composes with a cross-family serve: the pack is placed into the native body first, then that body is translated, so an open model served via openai_chat receives the playbook too.

{
  "enabled": true,
  "rules": [
    {
      "id": "writer-open-model-with-playbook",
      "match": { "call_type": "short_write", "api_key_id": "key_…" },
      "serve": { "model": "deepinfra/glm-5.3", "protocol": "openai_chat" },
      "inject": { "experiment_pack_id": "xpk_…" }
    }
  ]
}

The pack is never injected for a request in the standing holdout, when corrections consent is off, or when the bound pack is unavailable — each case is still served (translated, if cross-family) and recorded as a named skip under routing_decision.experiment_pack (ids and the module hash only, never the pack text).

Running several arms per workspace (per-key rules)

Add match.api_key_id to a rule so it fires only for requests made with that API key. Mint one key per arm and point each at the gateway: one key can be served by your frontier model untouched while another is served by an open model through translation, side by side in the same workspace.

Honesty in the record. When a rule rewrites a request:

• The response carries X-UR-Served-Model with the model actually called. The response body's model field is whatever the provider returned — it is never rewritten back.
• GET /v1/generation?id=req_… shows model_requested (your model) and model_served (the cheaper one), and routing_decision.passthrough_rule with the rule_id, both models, cache_injected and params_dropped. A cross-family serve also records routing_decision.family_swap ({ from_api, to_protocol, translation_version }); a refused translation records routing_decision.translation_refused.
• pnpm ledger:export carries the requested vs served model and the cache-read tokens per request, so a per-stage saving is readable straight from the export.

BYOK

If your workspace has a provider key configured for OpenAI or Anthropic, the passthrough uses it; otherwise it uses the platform key. Your own Authorization / x-api-key is never forwarded upstream — the platform or BYOK key is injected server-side into the provider's own auth header.