Import history

You do not have to wait for new traffic to get value from the Flywheel. POST /v1/imports brings your history in — a usage export from your provider dashboard, or intact request/ response traces with their outcomes — and returns an honest capability report before anything is analysed. Every row that cannot be consumed is rejected with a named reason, and a rejected row is repairable: fix it and resume with the same import id.

Import is deliberately conservative about what a file can prove. That conservatism is the feature.

Two import classes, and the difference is the point

The adapter you choose pins the import class, and the class caps what the import can ever claim:

• Usage-only (usage_summary) — per-model spend and exposure summaries. It reveals which models you send traffic to, what they cost, and which fields were missing. It can never manufacture a prompt, an output, a quality figure or a replayable case — its report is forced to replayable: false, regardless of what a file claims.
• Intact traces (e.g. openai_chat_jsonl) — full {request, response, occurred_at, outcome?} records. These additionally carry bodies, stored only under corrections_consent and redacted on ingest. Only then can candidate models be replayed against your real cases.

Imported records are always source: imported with exposure_arm: imported — they inform discovery and can seed distillation, but they are never a live-lift contrast. See Holdout and lift for why the arm matters.

Prerequisites

• The stack running. Locally, pnpm dev:e2e boots the gateway on http://localhost:8080. These endpoints are part of the learning preview, which is not on the invite-only hosted gateway (https://api.ultrarouter.io) yet — run the local stack or point at your own deployment.
• A key with outcomes:write (or an owner/admin session with flywheel.import). New keys are granted the scope by default; check with GET /v1/key.

export UR_BASE_URL="http://localhost:8080"
export UR_API_KEY="ur_sk_live_…"   # ur_sk_live_* or ur_sk_test_*

1. Preview before you commit

"mode": "preview" returns the full capability report with nothing stored — the honest way to see the class truth, the rejection counts and the missing fields before you ingest.

curl -sS -X POST "$UR_BASE_URL/v1/imports" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "format": "usage_summary", "mode": "preview",
    "records": [
      { "model": "openai/gpt-4o", "requests": 1200, "input_tokens": 480000,
        "output_tokens": 96000, "cost_usd": 18.40 }
    ]
  }'
# → { "import": { "import_class": "usage_only", "status": "preview",
#       "report": { "class_truth": "…usage, spend and missing fields; it cannot manufacture
#                    prompts, quality results or replayable cases", "replayable": false, … } } }

The report.class_truth sentence states, in plain language, exactly what this import can and cannot prove — and a usage-only import's is always the conservative one.

2. Commit, and read the per-row receipt

A commit ingests. Ingest is idempotent per record content hash (resume never double-ingests), and a malformed row is rejected with a named reason — never guessed. Below, one row is missing its model, so the good row is accepted and the bad one is reported:

# Import a two-row usage history where ONE row is malformed (no `model`). The good row is
# accepted; the bad one is rejected and NAMED in the receipt. Zero model calls.
imp=$(curl -sS --fail -X POST "$UR_BASE_URL/v1/imports" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d '{"format":"usage_summary","records":[
        {"model":"openai/gpt-4o","requests":12,"input_tokens":4800,"output_tokens":960,"cost_usd":0.21},
        {"requests":3,"input_tokens":900,"output_tokens":120,"cost_usd":0.04}]}')
id=$(printf '%s' "$imp" | python3 -c 'import sys,json; d=json.load(sys.stdin)["import"]; assert d["accepted_count"]==1 and d["record_count"]==2, d; assert d["import_class"]=="usage_only", d; print(d["id"])')
echo "ok: import $id accepted 1/2 rows; the malformed row is named in report.reasons"

# REPAIR: fix the row and resume with the SAME import_id. Idempotent per content hash, so
# the already-good row dedupes and only the repaired row lands — never double-counted.
curl -sS --fail -X POST "$UR_BASE_URL/v1/imports" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d "{\"format\":\"usage_summary\",\"import_id\":\"$id\",\"records\":[
        {\"model\":\"openai/gpt-4o\",\"requests\":12,\"input_tokens\":4800,\"output_tokens\":960,\"cost_usd\":0.21},
        {\"model\":\"openai/gpt-4o-mini\",\"requests\":3,\"input_tokens\":900,\"output_tokens\":120,\"cost_usd\":0.04}]}" \
  | python3 -c 'import sys,json; d=json.load(sys.stdin)["import"]; assert d["accepted_count"]>=1 and d["duplicate_count"]>=1, d; print("ok: repaired row accepted; the already-good row deduped")'

Expected state. The first call returns accepted_count: 1, the malformed row counted under rejected_count, and report.reasons maps the reason (a usage row requires a model) to a count. The repair call, resumed with the same import_id, accepts the fixed row and reports the unchanged row as duplicate — resume never double-counts.

3. Intact traces need consent

To bring full request/response bodies (the only thing that unlocks replay), use an intact-trace adapter and grant corrections_consent first. Without consent, a commit is refused; a preview still returns the capability report without storing a body:

# Preview first — no consent needed to see what WOULD be stored.
curl -sS -X POST "$UR_BASE_URL/v1/imports" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d '{ "format": "openai_chat_jsonl", "mode": "preview",
        "jsonl": "{\"request\":{\"model\":\"openai/gpt-4o\",\"messages\":[]},\"response\":{},\"occurred_at\":\"2026-06-01T00:00:00Z\"}" }'
# → report.import_class = "intact_trace", bodies_stored = 0 (preview never stores)

Grant consent on the Flywheel screen (owner-only, audited); revoking it stops new derivations and tombstones existing exports — it never rewrites what was already recorded.

4. Bind history to a call type so its verdicts can count

History with task verdicts ("outcome": { "channel": "resolution", "kind": "resolved_true" }) can move a workflow's readiness, the count of resolved outcomes a workflow needs before the Flywheel will test an improvement for it. For that, the import must say which call type the history belongs to. Name it with call_type_id (the ct_… id from GET /v1/call-types). The call type must have a confirmed objective: confirm its learning plan first. An objective that is only proposed never teaches.

Bringing history first. If the workspace has traffic but no outcome events yet, there is nothing to confirm the usual way — discovery only proposes the spend-leading workflow, and a confirm with no measured metric refuses no_feasible_objective. Break the circle with two session actions (owner / admin / developer, never an API key): propose the named workflow with POST /v1/flywheel/plans { "call_type_id": "ct_…" }, then confirm it choosing the metric your history populates — POST /v1/flywheel/plan/<plan_id>/confirm { "objective_metric": "claim_resolution" }. The confirm reports objective_source: "declared". Declared means confirmed, not measured: the plan compiles below teach until the channel is observed and qualified; choosing a metric fabricates no observation. For a workflow-execution import, claim_resolution is the metric its terminal decisions populate (delivered → resolved_true, held → resolved_false on the resolution channel). See Declare what "it worked" means for the full flow.

curl -sS -X POST "$UR_BASE_URL/v1/imports" \
  -H "Authorization: Bearer $UR_API_KEY" -H "Content-Type: application/json" \
  -d '{ "format": "openai_chat_jsonl", "call_type_id": "ct_01J…",
        "records": [
          { "request": { "model": "openai/gpt-4o",
                         "messages": [ { "role": "system", "content": "You triage support tickets." },
                                       { "role": "user", "content": "My order never arrived." } ] },
            "response": { "choices": [ { "message": { "content": "Refund issued." } } ] },
            "occurred_at": "2026-06-01T00:00:00Z",
            "outcome": { "channel": "resolution", "kind": "resolved_true" } } ] }'
# → { "import": { "call_type_id": "ct_01J…", "artifact_type_version_id": "atyp_01J…",
#       "report": { "binding_truth": "Bound to call type 'ct_01J…' (confirmed objective version …) …" } } }

Expected state. The import returns the call type it is bound to and the confirmed objective version it was bound under. report.binding_truth says, in one sentence, what that means. Each accepted record becomes an artifact of that call type, stamped with its confirmed objective and with the same situation your live traffic from that call site gets. The situation is derived from the request's shape: its instruction template, tools and response format. So history lands in the same workflow unit as the calls you make through the gateway.

What still stands between the upload and readiness. Each import batch is its own verdict source. A batch cannot prove on its own that its verdicts are right, so they stay held until someone who is not the source checks a sample of them. That check is a blind admission audit:

1. Open it with POST /v1/flywheel/source-audits and { "verdict_source": "import:<import_id>", "purpose": "admission" }. You must use a signed-in owner or admin session.
2. The response lists 40 sampled artifact ids, without the batch's verdicts. Label each one through the outcome API: a resolution event on the artifact.
3. Grade the audit with POST /v1/flywheel/source-audits/<id>/grade.

With zero mislabels the batch is admitted. From then on, its history counts toward that call type's readiness. After the next scheduler pass (every five minutes by default), GET /v1/flywheel/moat-next shows the new count, for example "needs 155 more resolved task outcomes (45/200)". Only history recorded up to the audit counts. Records added to the batch later are stamped, but they do not count on that audit.

Rules that keep a batch honest:

• A batch is bound once, when it is created. A resume (import_id) keeps that binding without repeating call_type_id. Naming a different call type on a resume is refused.
• If the call type's objective is re-confirmed, resuming an older batch is refused as stale. Start a new import; it binds to the current objective.
• A usage-only file cannot be bound. It creates no artifacts.
• An import without call_type_id is unbound. Its report says so, and its verdicts never count toward any workflow's readiness, even if the batch is admitted. Imports created before this feature are unbound too, and they are never bound after the fact. To bind that history, upload the same file again as a new import naming call_type_id. The new batch creates its own bound artifacts next to the old ones and needs its own admission audit.

Re-confirming changes the measurement — and your bound history cannot follow it today. Re-confirming a workflow with a changed measurement mints a new confirmed objective version, and history bound to the old version goes stale against it (that is the law, not a bug: a new measurement must not silently re-score old evidence). There is no re-import recovery for that history today. For a usage or intact-trace (openai_chat_jsonl) import, whose artifact identity is the record content hash, you can upload the same file again as a new import and it binds to the current objective. But a workflow-execution import (workflow_execution_jsonl) groups your gateway's own request rows, and the first import already owns those member requests — re-importing the same executions under the new objective is refused request_already_grouped (the membership law: one request belongs to one active artifact). So choose the metric deliberately at the first confirm: a later measurement change leaves the bound batch stale, and the same executions will not re-import under it. (A lawful versioned re-analysis that preserves an unchanged measurement is a planned capability, not a promise here.)

Name call types the way your traffic does. In a workflow_execution_jsonl record, lever_call_type and each call's call_type may be the ct_… id, the slug you send in X-UR-Call-Type, or one of its aliases. The importer resolves each name to the id before it checks the lever against the batch's call_type_id and each call against its stored request, exactly as the gateway resolves your header. A name that matches no call type in the workspace is refused by name (lever_call_type_mismatch / request_call_type_mismatch).

5. Cancel a non-terminal import

A preview or running import can be cancelled. Cancellation releases nothing to spend (imports reserve no budget) and preserves per-record idempotency, so a later resume picks up cleanly:

curl -sS -X POST "$UR_BASE_URL/v1/imports/imp_01J…/cancel" \
  -H "Authorization: Bearer $UR_API_KEY"
# → { "import": { "status": "cancelled", … } }   ·  a terminal import is a 409

The dashboard receipt

The Flywheel screen renders the same receipt: each rejected row's state is named, and a reviewer event that could not be joined (a mismatched draft version) is a first-class dead letter you re-drive with POST /v1/flywheel/reviewer/repair. A repair re-runs the stored event; it never rewrites a recorded consequence.

Troubleshooting

What this proves — and what it does not

Proves. You can bring history in on Day 1 without touching your traffic — usage exposure and spend with no consent, redacted intact traces with consent — with the class truth stated up front, per-record idempotency, and every rejected row named and repairable. History bound to a confirmed call type and admitted by a blind audit counts toward that workflow's readiness, in the same unit as its live traffic.

Does not prove. A usage-only import proves nothing about quality — it cannot manufacture a prompt or a result. An intact-trace import gives you replayable cases, not a verdict; the verdict is a separate, budgeted analysis you authorize. Imported data is exposure_arm: imported and is never a live-lift contrast. An admission audit bounds the batch's error rate, at roughly 9 % after zero mislabels in 40 samples. It does not prove every verdict right. Readiness only decides when an improvement is worth testing; the improvement still has to pass its own screen and a live comparison.

Next steps

• Record outcomes — the live capture path the import backfills.
• Plan a model migration — where a usage import becomes a scoped decision.
• The training-ready export — what leaves the ledger as a dataset you own.