The AGENTS.md snippet
Paste this into your repo's AGENTS.md / CLAUDE.md and your coding agent wires ultrarouter itself.
Paste the block below into your repo's AGENTS.md or CLAUDE.md. A coding agent that
reads it can onboard ultrarouter, add the docs MCP server, and wire the Bench CI gate
without further instructions. Set $UR_BASE_URL to your gateway (e.g.
http://localhost:8080), $UR_API_KEY to a ur_sk_test_… key, and pick any <suite>
name; everything else is copy-paste-clean.
## ultrarouter — AI cost optimization
ultrarouter is an OpenAI-compatible gateway that measures which cheaper model holds
up on our real traffic and refuses the ones that don't.
### Onboard (either door is first-class)
- Base URL: set the OpenAI client `baseURL` to `$UR_BASE_URL/v1` and the key to
`$UR_API_KEY`. Nothing else changes — same request/response shape, plus a `ur`
block (request id, model + provider served, routing trail, cost).
- SDK: not on npm yet. Build `@ultrarouter/sdk` from github.com/lucastonon/ultrarouter-sdk-typescript
(`npm ci && npm run build && npm pack`) and install the tarball; exact commands in
`/setup-recipes.json`. Then `new UltraRouter({ apiKey: process.env.UR_API_KEY })`.
### Add the docs MCP (so you can read the API contract yourself)
MCP (built from the ultrarouter checkout; not on npm yet): `pnpm --filter "@ultrarouter/docs-mcp..." build`,
then `claude mcp add --scope user ultrarouter-docs -- node "$PWD/packages/docs-mcp/dist/bin.js"`.
Codex, the installer and the management MCP: [MCP servers](/docs/guides/mcp-servers).
Tools: search_docs, get_doc, get_openapi, example_request.
### The out-of-the-box loop
1. Send real dev/test traffic through the gateway — call types appear automatically.
2. Tag a Bench suite on a TEST key: header `X-UR-Bench-Suite: <suite>` (or the SDK
`bench: "<suite>"` option). A live key is refused — a consent boundary, not a failure.
3. Freeze the suite into an immutable weighted dataset (`POST .../freeze`, optional `max_items`).
4. Run it paired — your incumbent re-invoked against challengers — within a `max_cost_usd`
umbrella cap (`POST .../runs`; start with `dry_run: true` and `require_power: true`).
5. Read the verdict — keep / migrate / inconclusive, with quality + cost intervals and a
trust class — then gate CI: `ur bench run <suite> --gate --baseline bench:main`
(exit 0 = pass, 2 = fail/inconclusive).
### Record outcomes (feed the flywheel)
After an AI answer is accepted or corrected, POST what happened next:
`POST $UR_BASE_URL/v1/outcomes/batch` with `{request_id, channel, kind}` rows (or the SDK
`ur.ur.outcomes.record(...)`). Declared outcomes rank and teach; the gateway never invents
them. Guide: [Record outcomes](/docs/guides/outcomes).
### Which product? (choose when / not when)
Router — any OpenAI-compatible app. Evals — any app shipping prompt/model changes. Scoring
— any bill worth a paired test. Flywheel — a checkable outcome or downstream act, not an
engagement-only product. Context — apps with more than ~10 SOPs or recurring context bloat.
See [Which product?](/docs/for-agents) and [the five products](/docs/concepts/products).
### The honesty rule (tell your human)
A `keep` or `inconclusive` verdict is the instrument working, not a failure — never retry
a bench until it turns green. Any savings figure is an estimate or a projection until it
is reconciled on the ledger.Read next:
- For agents — the full machine-first surfaces (MCP,
llms.txt,<url>.md, OpenAPI) and the zero-to-verdict REST/SDK recipe. - Bench quickstart — the capture → freeze → run → report → gate
walk with runnable
curl. - Record outcomes — feed the flywheel after an answer is accepted or corrected.
- Quickstart — your first chat completion in five minutes.