Record outcomes

Once you have declared what "it worked" means for a kind of artifact, the outcome ledger records what actually happened to each one — a user accepting a draft, a claim resolving true against a filing months later, a nightly job backfilling thousands of resolutions from your warehouse. This guide is the worked example from the spec (the investment-thesis case), showing the five ways an outcome reaches the ledger, in TypeScript, Python and curl side by side.

Two properties hold for every path below and are worth stating once:

• Provenance is the server's. An outcome event has no source, producer, learning_use or latency_class field — those are assigned from your credential and the rule that fired. A caller-supplied value is a 400, never a silent strip, so a recorded signal always means exactly what its provenance says.
• It is never a body. Outcome events carry ids, hashes, lengths and typed numeric payloads only. The one route that may carry a body is /final, and only under corrections_consent — covered in the correction quickstart.

Prerequisites

• The stack running. Locally, pnpm dev:e2e boots the gateway on http://localhost:8080. These endpoints are part of the learning preview, which is not on the invite-only hosted gateway (https://api.ultrarouter.io) yet — run the local stack or point at your own deployment.
• A key with the outcomes:write scope. New keys are granted it by default; check with GET /v1/key. Use a ur_sk_test_* key to rehearse — test-mode data is captured but never enters learning, invoicing or your allowance.

export UR_BASE_URL="http://localhost:8080"
export UR_API_KEY="ur_sk_live_…"   # ur_sk_live_* or ur_sk_test_*

0. Declare the objective (once)

Every artifact of a type pins the objective version current when it was created, so declare it before you record. See Declare what "it worked" means for the full field table; the thesis case is a forecast whose claims resolve true or false against a company filing, within twelve months.

curl -sS -X POST "$UR_BASE_URL/v1/artifact-types" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "name": "thesis", "archetype": "forecast",
    "objective": { "metric": "claim_resolution", "unit": "boolean", "direction": "true_is_good",
      "resolution_criteria": "company filing vs stated claim", "source": "analyst", "window": "P12M" }
  }'

import UltraRouter from "@ultrarouter/sdk";

const ur = new UltraRouter({
  apiKey: process.env.UR_API_KEY,
  baseURL: process.env.UR_BASE_URL, // http://localhost:8080 locally
});

// The flywheel surfaces live under `ur.ur.*`; chat/embeddings/models are top-level.
await ur.ur.artifactTypes.upsert({
  name: "thesis", archetype: "forecast",
  objective: { metric: "claim_resolution", unit: "boolean", direction: "true_is_good",
    resolution_criteria: "company filing vs stated claim", source: "analyst", window: "P12M" },
});

from ultrarouter import UltraRouter

client = UltraRouter()  # reads UR_API_KEY + UR_BASE_URL from the environment

# The flywheel surfaces are flattened to the Python idiom (client.artifacts, …).
client.artifact_types.upsert(
    name="thesis", archetype="forecast",
    objective={"metric": "claim_resolution", "unit": "boolean", "direction": "true_is_good",
               "resolution_criteria": "company filing vs stated claim",
               "source": "analyst", "window": "P12M"},
)

Then create the artifact — the unit of value outcomes attach to — declaring its claims:

curl -sS -X POST "$UR_BASE_URL/v1/artifacts" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "key": "thesis:AAPL-2026Q4", "archetype": "forecast", "artifact_type": "thesis",
    "claims": [{ "key": "eps_beat", "claim_type": "earnings_direction",
      "stated_confidence": 0.7, "resolves_by": "2026-12-31T00:00:00Z" }]
  }'

const art = await ur.ur.artifacts.create({
  key: "thesis:AAPL-2026Q4", archetype: "forecast", artifact_type: "thesis",
  claims: [{ key: "eps_beat", claim_type: "earnings_direction",
    stated_confidence: 0.7, resolves_by: "2026-12-31T00:00:00Z" }],
});

art = client.artifacts.create(
    key="thesis:AAPL-2026Q4", archetype="forecast", artifact_type="thesis",
    claims=[{"key": "eps_beat", "claim_type": "earnings_direction",
             "stated_confidence": 0.7, "resolves_by": "2026-12-31T00:00:00Z"}],
)

Your declared claims always win their key

The platform can also extract claims from a consented output (their keys start with ext:; they show provenance: "extracted" on the artifact). Your declared claims outrank them, in either order:

• You declared first. A later extraction of the same claim_key writes nothing; your claim is untouched.
• The extraction came first. When you then declare the same claim_key, your claim becomes the key's active claim (provenance: "declared", qualification: "declared"). The extracted claim is not deleted: it moves to the artifact's superseded_claims, with state: "superseded" and superseded_by_claim_id pointing at your claim. Outcomes already recorded on it stay on it, and they do not count toward yours. From then on a claim_key outcome lands on your claim.
• Never the reverse. An extraction never replaces a claim you declared. Re-declaring a key you already declared keeps your first declaration (the upsert is idempotent per key).

1. Instant — the moment the user acts

An instant outcome is recorded within five minutes of the artifact's occurred_at; the server derives latency_class: "instant" for you. This is the user accepting the draft the moment it appears — a behavioral signal that ranks but never teaches.

curl -sS -X POST "$UR_BASE_URL/v1/artifacts/art_01J…/outcomes" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d '{ "outcomes": [{ "channel": "behavioral", "kind": "accepted" }] }'
# → 202 { "object": "list", "data": [{ "status": "accepted", "latency_class": "instant",
#         "source": "declared", "learning_use": ["rank"], … }] }

// The SDK default observed_at = now, so this is an instant outcome.
await ur.ur.outcomes.record(art.id, { channel: "behavioral", kind: "accepted" });

receipt = client.outcomes.record(art.id, {"channel": "behavioral", "kind": "accepted"})

2. Delayed — the resolution that arrives months later

A delayed outcome carries an observed_at more than five minutes after the artifact's occurred_at; the server derives latency_class: "delayed". This is the filing landing next quarter and resolving the eps_beat claim — a resolution signal that teaches. observed_at may be earlier than now (a historical resolution) but must be at or after the artifact's occurred_at.

curl -sS -X POST "$UR_BASE_URL/v1/artifacts/art_01J…/outcomes" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d '{ "outcomes": [{ "claim_key": "eps_beat", "channel": "resolution", "kind": "resolved_true",
        "observed_at": "2027-01-30T00:00:00Z", "source_event_id": "filing:Q4" }] }'
# → 202 { …, "data": [{ "status": "accepted", "latency_class": "delayed",
#         "source": "declared", "learning_use": ["calibrate", "train"], … }] }

await ur.ur.outcomes.record(art.id, {
  claim_key: "eps_beat", channel: "resolution", kind: "resolved_true",
  observed_at: "2027-01-30T00:00:00Z", source_event_id: "filing:Q4",
});

client.outcomes.record(art.id, {
    "claim_key": "eps_beat", "channel": "resolution", "kind": "resolved_true",
    "observed_at": "2027-01-30T00:00:00Z", "source_event_id": "filing:Q4",
})

The source_event_id is the idempotency discriminator. The idempotency key is (producer, target, kind, source_event_id): an identical replay is a duplicate no-op; the same key with a different payload is a 409 conflict and nothing is written. That is why a re-run is always safe.

3. Batch — up to 1,000 events keyed by target

A batch records up to 1,000 events in one call, each keyed by request_id, artifact_id or artifact_key (plus an optional claim_key). Every row gets its own verdict (accepted | duplicate | conflict | rejected) and the response carries a rollup.

curl -sS -X POST "$UR_BASE_URL/v1/outcomes/batch" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d '{ "outcomes": [
        { "artifact_key": "thesis:AAPL-2026Q4", "channel": "mechanical", "kind": "schema_valid" },
        { "artifact_key": "thesis:MSFT-2026Q4", "claim_key": "eps_beat",
          "channel": "resolution", "kind": "resolved_false" } ] }'
# → { …, "accepted": 2, "duplicate": 0, "conflict": 0, "rejected": 0, "pending": 0 }

await ur.ur.outcomes.batch([
  { artifact_key: "thesis:AAPL-2026Q4", channel: "mechanical", kind: "schema_valid" },
  { artifact_key: "thesis:MSFT-2026Q4", claim_key: "eps_beat",
    channel: "resolution", kind: "resolved_false" },
]);

client.outcomes.batch([
    {"artifact_key": "thesis:AAPL-2026Q4", "channel": "mechanical", "kind": "schema_valid"},
    {"artifact_key": "thesis:MSFT-2026Q4", "claim_key": "eps_beat",
     "channel": "resolution", "kind": "resolved_false"},
])

A row keyed by request_id follows the receipt contract of the request route: it carries its receipt, and when that request has not been recorded yet the row is pending (counted in the rollup's pending) and is applied once it is. A retry of the same row — through the batch or POST /v1/requests/{id}/outcomes — is the same receipt. Refusals stay in-row: a rejected row names request_not_found or outcome_backlog_full; the batch itself still answers 200. Rows keyed by artifact_id / artifact_key carry receipt: null.

4. Warehouse backfill — the ur outcomes CLI

You do not have to thread ids by hand to bring resolutions in from a data warehouse. The ur outcomes import CLI reads a JSONL or CSV export and drives the same idempotent batch route. When a row omits source_event_id, the CLI derives a deterministic one from the row's content, so re-importing the same file is all duplicates — re-runs are free. It cannot smuggle provenance: source, producer and learning_use columns in your file are dropped, never forwarded.

# Export from your warehouse to JSONL (or CSV), one outcome per row:
#   {"artifact_key":"thesis:AAPL-2026Q4","claim_key":"eps_beat","channel":"resolution","kind":"resolved_true"}
ur outcomes import ./resolutions.jsonl          # idempotent; --map target=source to remap columns
ur outcomes pending --state overdue             # claims past resolves_by, still unresolved

The CLI's exit-code contract is strict: 0 when every row is accepted or a duplicate; 2 on a partial rejection with each rejected row listed (client-side parse errors and server rejected/conflict rows alike); 2 fail-closed on any read or API error. A broken backfill is never silently green.

5. Flush — the durable queue at end of run

The SDK's record() durably persists each event before the first send and replays it verbatim on retry, so a transient network failure can only ever produce a duplicate, never a conflict. At the end of a run, drain the queue:

import UltraRouter, { FileOutcomeStore } from "@ultrarouter/sdk";

// Cross-restart durability is opt-in; the default queue is in-process.
const ur = new UltraRouter({ outcomeStore: new FileOutcomeStore("queue.json") });

// A scope binds ur.artifact on every call inside it and flushes on exit (success or throw):
await ur.withArtifact("thesis:AAPL-2026Q4", async (scope) => {
  await scope.chat.completions.create({ model: "openai/gpt-5-mini",
    messages: [{ role: "user", content: "…" }] });
  await scope.record({ channel: "resolution", kind: "resolved_true", claim_key: "eps_beat" });
});

await ur.ur.outcomes.flush(); // drain the durable queue at end-of-execution

from ultrarouter import UltraRouter, FileOutcomeStore

client = UltraRouter(outcome_store=FileOutcomeStore("queue.json"))  # default is in-process

with client.with_artifact("thesis:AAPL-2026Q4") as scope:
    scope.chat.completions.create(model="openai/gpt-5-mini",
                                  messages=[{"role": "user", "content": "…"}])
    scope.record({"channel": "resolution", "kind": "resolved_true", "claim_key": "eps_beat"})

client.outcomes.flush()  # drain the durable queue at end-of-run

6. Report the moment you have the answer — receipts

The request shortcut POST /v1/requests/{id}/outcomes joins an outcome to one gateway request by its x-ur-request-id. The gateway writes that request's record after it has answered you, so an outcome reported the instant your app holds the final response can arrive first. That is fine: every 202 carries a durable receipt, and you never wait on — or read — anything but the receipt.

# Optional: mint the request id yourself (`[A-Za-z0-9_-]{1,64}`) so you know it before the call.
curl -sS "$UR_BASE_URL/v1/chat/completions" -H "Authorization: Bearer $UR_API_KEY" \
  -H "X-UR-Request-Id: run-42-step-7" -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-5-mini","messages":[{"role":"user","content":"…"}]}'

# Report as soon as your own check has run — even before the response above returns.
curl -sS -X POST "$UR_BASE_URL/v1/requests/run-42-step-7/outcomes" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d '{"channel":"computable","kind":"oracle_pass","source_event_id":"run-42-verdict"}'
# → 202 { "object": "outcome", "status": "pending", "id": null, …,
#         "receipt": { "object": "outcome_receipt", "id": "ocr_…", "status": "pending", … } }

# The receipt resolves on its own; read it whenever you like.
curl -sS "$UR_BASE_URL/v1/outcome-receipts/ocr_…" -H "Authorization: Bearer $UR_API_KEY"
# → { "status": "applied", "outcome_event_id": "oev_…", "outcome_status": "accepted",
#     "episode": { "status": "associated", "trace_id": "ept_…", "verdict": "recorded" }, … }

The top-level status. accepted — recorded now; duplicate — a retry of an outcome already recorded (nothing changes, nothing is counted twice); pending — the request has not been recorded yet: the outcome is stored durably and applied when it is.

The receipt. GET /v1/outcome-receipts/{id} reads one; GET /v1/requests/{id}/outcome-receipts lists every receipt for a request (an empty list when you reported nothing — an outcome nobody sent stays unknown, it is never inferred).

Retries and conflicts. The idempotency key is (your credential, request id, kind, source_event_id). Re-sending the identical outcome — after a lost acknowledgement, a timeout, a crash — returns the same receipt: pending while pending, duplicate once applied. Reusing the key with a different payload is 409 conflict. Without a source_event_id the server mints one, so every send is a new outcome — always send one when you retry.

Bounds. A pending receipt waits at most 15 minutes for its request, then expires request_not_found — a request id of another workspace and an id that never existed are indistinguishable. A gateway-minted req_… id carries its own mint time, so one minted more than 15 minutes ago (or more than 2 minutes in the future) with no request is refused at once (404, nothing stored). A workspace may hold 5,000 pending receipts; beyond that the report is 429 outcome_backlog_full with a Retry-After hint.

Revisions and late outcomes. The first teaching verdict on an episode wins; a later, different outcome is recorded beside it (kept_earlier) and never silently overrides it. To change your mind, send the new outcome with supersedes_event_id naming the earlier one — its oev_… id or its source_event_id — from a key of equal or higher authority (a workspace owner/admin session outranks an API key). Before any learned version has used the episode the verdict is replaced (revised); afterwards the revision is recorded and counted as revised_after_incorporation, and the learned version is not silently unlearned. A revision that arrives before the outcome it supersedes simply holds the verdict; the late original is kept out.

Concurrent episodes with the same opening. Every step that carries the same X-UR-Artifact header (or body ur.artifact) belongs to one episode. Without it, steps are grouped by their opening (system prompt + first user message), so two tasks running at the same time with identical openings are treated as one episode and the first verdict wins. Give each concurrent task its own handle to keep their episodes — and their verdicts — apart.

Run the whole thesis loop against your stack

This self-contained example does the full walkthrough as raw REST — objective, an instant behavioral outcome, a delayed resolution, a batch, and a read of pending claims — asserting the server-derived latency_class at each step. It makes zero paid model calls (outcomes move metadata, not prompts):

# 0. Declare the objective (idempotent — re-running is a no-op that returns the same version).
curl -sS --fail -X POST "$UR_BASE_URL/v1/artifact-types" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d '{"name":"thesis","archetype":"forecast","objective":{"metric":"claim_resolution","unit":"boolean","direction":"true_is_good","resolution_criteria":"company filing vs stated claim","source":"analyst","window":"P12M"}}' \
  > /dev/null
echo "ok: objective 'thesis' declared"

# An artifact whose occurred_at is NOW → an outcome recorded now is INSTANT.
akey="outcomes-quickstart:$(date +%s)-$RANDOM"
aid=$(curl -sS --fail -X POST "$UR_BASE_URL/v1/artifacts" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d "{\"key\":\"$akey\",\"archetype\":\"forecast\",\"artifact_type\":\"thesis\"}" \
  | python3 -c 'import sys,json; print(json.load(sys.stdin)["artifact"]["id"])')

# 1. Instant — behavioral/accepted (ranks, never teaches); server derives latency_class=instant.
curl -sS --fail -X POST "$UR_BASE_URL/v1/artifacts/$aid/outcomes" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d '{"outcomes":[{"channel":"behavioral","kind":"accepted"}]}' \
  | python3 -c 'import sys,json; r=json.load(sys.stdin)["data"][0]; assert r["latency_class"]=="instant", r; assert r["source"]=="declared", r; print("ok: instant outcome —", r["learning_use"])'

# An artifact whose occurred_at is in the PAST, with a claim to resolve later.
bkey="outcomes-thesis:$(date +%s)-$RANDOM"
bid=$(curl -sS --fail -X POST "$UR_BASE_URL/v1/artifacts" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d "{\"key\":\"$bkey\",\"archetype\":\"forecast\",\"artifact_type\":\"thesis\",\"occurred_at\":\"2026-01-01T00:00:00Z\",\"claims\":[{\"key\":\"eps_beat\",\"claim_type\":\"earnings_direction\",\"stated_confidence\":0.7,\"resolves_by\":\"2026-12-31T00:00:00Z\"}]}" \
  | python3 -c 'import sys,json; print(json.load(sys.stdin)["artifact"]["id"])')

# 2. Delayed — the resolution lands months after occurred_at; server derives latency_class=delayed.
curl -sS --fail -X POST "$UR_BASE_URL/v1/artifacts/$bid/outcomes" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d '{"outcomes":[{"claim_key":"eps_beat","channel":"resolution","kind":"resolved_true","observed_at":"2026-06-01T00:00:00Z","source_event_id":"filing:Q2"}]}' \
  | python3 -c 'import sys,json; r=json.load(sys.stdin)["data"][0]; assert r["latency_class"]=="delayed", r; assert "train" in r["learning_use"], r; print("ok: delayed resolution —", r["learning_use"])'

# 3. Batch — two events keyed by artifact_key, each with its own verdict + a rollup.
curl -sS --fail -X POST "$UR_BASE_URL/v1/outcomes/batch" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d "{\"outcomes\":[{\"artifact_key\":\"$akey\",\"channel\":\"mechanical\",\"kind\":\"schema_valid\",\"source_event_id\":\"b1\"},{\"artifact_key\":\"$bkey\",\"channel\":\"mechanical\",\"kind\":\"tool_ok\",\"source_event_id\":\"b2\"}]}" \
  | python3 -c 'import sys,json; d=json.load(sys.stdin); assert d["accepted"]==2, d; print("ok: batch accepted", d["accepted"], "events")'

# 4. Pending claims — a keyset list of what is still awaiting resolution.
curl -sS --fail "$UR_BASE_URL/v1/outcomes/pending" \
  -H "Authorization: Bearer $UR_API_KEY" \
  | python3 -c 'import sys,json; d=json.load(sys.stdin); assert d["object"]=="list", d; print("ok: pending claims listed —", len(d["data"]), "shown")'

Non-proxied calls — the exposure grant

If you call the model directly (not through the gateway proxy), the artifact you record never passed a gateway dispatch, so it is an unarmed_observation: its outcomes are captured and visible but stay out of the randomized learning contrast. The exposure grant arms it. Mint a grant before your model call, then quote it on the artifact upsert:

# 1. Mint the grant for the artifact you are about to produce (one of artifact_key | trace_id is
#    REQUIRED). The grant is deterministic + idempotent — the same key in the same epoch always
#    returns the same grant, and it is never cached, so mint it per call.
curl -sS http://localhost:8080/v1/flywheel/grants \
  -H "Authorization: Bearer $UR_KEY" -H 'Content-Type: application/json' \
  -d '{ "call_type": "thesis", "artifact_key": "thesis:AAPL-2026Q4" }'
# → 200 { "object": "exposure_grant", "grant_id": "grant_…", "arm": "treated",
#         "lever_id": null, "epoch": 7, "valid_until_epoch_close": "…", "request_id": "req_…" }

# 2. Call your model yourself (any provider) and build your artifact.

# 3. Upsert the artifact quoting the grant. `attestation` is your own attestation of what happened:
#    "served" = the grant was issued; "applied" = you actually used the recommended module/model.
#    A valid grant stamps the artifact with the grant's arm; an unknown grant is a 400.
curl -sS http://localhost:8080/v1/artifacts \
  -H "Authorization: Bearer $UR_KEY" -H 'Content-Type: application/json' \
  -d '{ "key": "thesis:AAPL-2026Q4", "archetype": "forecast",
        "grant_id": "grant_…", "attestation": "applied" }'
# → 201 { "artifact": { …, "exposure_arm": "treated", "grant_id": "grant_…",
#                       "grant_attestation": "applied" } }

# 4. Record outcomes exactly as above (steps 1–5). They now count under the grant's arm.

arm, epoch and grant_id are the server's — assigned by the same code that stamps a proxied request. attestation is yours: it is shown beside lift, never folded into provenance, and never used to select or weight an observation. An artifact you upsert without a grant stays unarmed_observation — arming is always your explicit, per-artifact choice.

Troubleshooting

What this proves — and what it does not

Proves. You can record every kind of outcome your product generates — instant or delayed, one at a time or a thousand at once, from the SDK or a warehouse export — with provenance assigned by the server, durable idempotent retries, and never a body. The latency_class, source and learning_use on each result are the server's honest classification of the signal.

Does not prove. Recording outcomes does not improve your app. A behavioral or proxy outcome ranks but never teaches; a resolution outcome is a joined observation, not a learned consequence. At M1 the learning block is unavailable / collecting, and no quality or cost-per-task claim is made here. Turning these outcomes into a tested, live-confirmed improvement is M2.

Once the flywheel is on, each workflow's Self-improve setting defaults to Assist — improvements that pass a live confirmation are pushed to your Improvements inbox for a human to adopt. You can switch a workflow to Off (capture only) or opt in to Autopilot (adopt within guardrails automatically); see The Flywheel.

Next steps

• Declare what "it worked" means — the field table behind step 0.
• Capture your first correction — the /final channel-2 path and the reviewer adapter.
• Import history — bring past traffic and resolutions in bulk, with repairable receipts.
• The Flywheel — how these outcomes become learning, behind a holdout.