Confirm a candidate in your own app

The Flywheel has found a repeated correction in your traffic and screened a candidate that might fix it. Before that candidate is allowed anywhere near your live requests, it has to clear evidence you can inspect: your own application tests, in your own CI, then a randomized live contrast against a frozen baseline. This M2 quickstart walks that path using only shipped surfaces — the /v1/flywheel/candidates/:ref/rollout gate, the signed decision callback, and the Flywheel screen — and it tells you plainly where the current build stops short of a button, so you always know which state you are looking at and why.

It is bounded on purpose. The runnable example below calls the rollout gate, which reads your workspace state and reports readiness. It makes zero paid model calls, mutates nothing, and returns in one request.

What you get — and what you do not yet

M2 gives you, without changing how a single live request is served:

• a rollout gate — one server-side function that reads the four facts your CI cannot see (permissions, consent, budget, qualification) and reports whether a controlled rollout may start, refusing with a named reason for each blocker;
• a staged candidate view on the Flywheel screen — offline screening, your application CI, and live confirmation shown as three distinct blocks, never merged into one verdict, plus a single derived progression headline;
• a signed, replay-safe decision record — a human accept/reject/revert, written as an audited receipt and never as a learning label;
• the adoption lease and rollback lifecycle — an adopted lever expires by default and reverts to the frozen baseline unless it is renewed.

What M2 does not yet give you:

The three evidence kinds are never blended

This is the load-bearing honesty rule of the whole page, and the code enforces it mechanically (@ultrarouter/core/flywheel/staged-evidence.ts):

• Offline screening is our replay on a disjoint split. A screening pass is not a live result.
• Application CI is your test/replay harness, in your CI. A passed check is not a live confirmation — and a direct model-call comparison can never claim application-level coverage.
• Live confirmation is a randomized contrast against a frozen baseline. It is the only evidence that can confirm improvement.

The three are derived from disjoint run stages, so an application check can never be presented as a live confirmation, and a screened candidate can never be presented as confirmed. "Passing CI, a merged PR or a model-generated correction is not a business-outcome label" is not a slogan here — it is where the type boundaries are.

Prerequisites

• The stack running. Locally, pnpm dev:e2e boots the gateway on http://localhost:8080 and the dashboard on http://localhost:3000. These endpoints are part of the learning preview, which is not on the invite-only hosted gateway (https://api.ultrarouter.io) yet — run the local stack or point at your own deployment.
• A key and base URL. Set them once:

export UR_BASE_URL="http://localhost:8080"
export UR_API_KEY="ur_sk_live_…"   # ur_sk_live_* or ur_sk_test_*

• The outcomes:write scope on that key — the rollout gate and the decision callback are Flywheel writes. New keys are granted it by default; confirm yours with GET /v1/key (step 1).
• A screened candidate. You reach this guide once the Flywheel has screened a learned_context candidate for one of your workflows. Until you have your own, every request below still returns a well-formed, honest response — the gate reports blockers for a candidate reference it has never seen rather than erroring.
• The Flywheel screen enabled. It is behind the dashboard.flywheel feature flag (an unknown flag reads off, and /flywheel returns 404 until it is on). It is seeded on in dev/e2e.
• Time and cost. About five to ten minutes. The gate call and the introspection call replay no prompts and make no model calls — they cost nothing.

1. Confirm your setup

GET /v1/key describes the key itself — its scopes and caps — so a 200 proves your base URL and key work end to end. Check that scopes includes outcomes:write.

curl -sS --fail "$UR_BASE_URL/v1/key" \
  -H "Authorization: Bearer $UR_API_KEY"
echo
echo "ok: key introspection reachable — confirm 'scopes' lists outcomes:write above"

2. Bind the candidate into your application CI

Application CI is the ultrarouter GitHub Action / CLI that detects a configured prompt, model, tool/schema, retrieval or Context change, binds it to your workflow IDs and your own application test command, and runs your baseline and candidate revisions through your own test/replay harness, in your own CI. The candidate arrives as an immutable reference (@ultrarouter/core#candidateReference), so the Flywheel row, the pull request and the harness configuration all point at the same object.

Three things about this step are guarantees, not conveniences:

• Your CI produces the headline, out of band. The gateway records the bound evidence-object hash; the pass/fail lives on your pull request. The headline is one of exactly three: Ready for controlled rollout, Needs evidence, or Regression found. A flaky test, a missing secret, a missing evaluator or an absent replay capability produces a named non-success — never success by omission.
• Untrusted code never sees platform secrets. Fork pull requests receive no secrets and no privileged execution of their code. Repository write is a separate grant from reading checks — you can run the whole workflow with only a downloadable patch and grant write later.
• Nothing production mutates during CI. A gateway-owned module change rides as an immutable candidate reference in the harness config; a code-owned change is a file diff you review before anything is written.

This step runs in your repository's CI, so there is no ultrarouter endpoint to curl here. See the Flywheel API reference for the exact evidence-object shape the gate reads back in step 3.

3. Check the rollout gate (the runnable bounded example)

Once your CI has a headline, ask the server whether a controlled rollout may start. The gate is the one server-side implementation (@ultrarouter/core#evaluateRolloutPreconditions): your CI supplies the facts only it observes (the bound identity + the headline), and the server adds the four facts your CI cannot see. It always returns 200 with pressed: false and the full list of blockers — you should see everything you must fix, not just the first.

curl -sS --fail -X POST \
  "$UR_BASE_URL/v1/flywheel/candidates/urcand1_docdemo/rollout" \
  -H "Authorization: Bearer $UR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "schema": "ur.flywheel.rollout.request.v1",
    "candidate_ref": "urcand1_docdemo",
    "candidate_state": "screened",
    "observed_identity": {
      "repository": "acme/app", "baseSha": "b", "headSha": "h",
      "lockDigest": "l", "harnessHash": "hh", "configHash": "ch",
      "datasetSplit": "holdout-a", "workflowId": "wf", "objective": "acc",
      "evaluatorVersion": "v3", "evidenceEpoch": "e1"
    },
    "application_headline": "ready_for_rollout",
    "requires_repository_write": false,
    "change_shape": "gateway_owned"
  }'
echo
echo "ok: the gate returned pressed:false with a named blocker list"

Expected state. A 200 whose body is { "unlocked", "pressed": false, "blockers": [...], "evidence_hash", "candidate_ref" }. pressed is the literal false — the gate can describe an unlocked rollout but has no representation for a started one. unlocked is true only when blockers is empty. On a fresh workspace you will typically see consent_revoked and budget_exceeded until you grant corrections consent and a confirmation slot is free — each blocker names its own family and reason.

The evidence_hash is the immutable evidence object the dashboard row and the pull request both link to — recomputable from the bound identity, so "the dashboard and the PR point at the same object" is a recomputation, not a promise.

4. What the gate refuses, and why

Every refusal is named and distinct, because your next action differs for each. There is no generic "blocked", and there is no input whose absence is skipped — a fact the server could not read is its own *_unknown refusal, never an implicit pass.

Try it: change application_headline to regression_found in step 3 and the gate adds application_check_not_passed; set requires_repository_write to true with a code_owned change shape and it adds repository_write_not_granted. The gate still returns 200, still pressed: false — it is telling you what to fix.

5. Record the human decision

When a person approves or refuses a candidate, that decision is written as a signed, replay-safe callback and stored as an audited receipt with the actor and a mandatory reason. It is deliberately never a learning label: a human's opinion of a candidate is not an independently verified outcome, so its learning_use is hard-wired to audit.

# Illustrative — a real call is SIGNED (X-UR-AppCI-Signature / X-UR-AppCI-Timestamp,
# the FLW-26 MAC recipe keyed by your API key). Without a valid signature the server
# FAILS CLOSED with 403 "missing_signature"; unauthenticated is 401. The signed path
# is exercised in the gateway e2e (E2E-FLW-ROLLOUT / TST-FLW-CONFIRMATION).
curl -sS -X POST \
  "$UR_BASE_URL/v1/flywheel/candidates/urcand1_docdemo/decisions" \
  -H "Authorization: Bearer $UR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "schema": "ur.appci.callback.v1", "kind": "decision",
    "callback_id": "urcb1_…", "workspace_id": "ws_…",
    "candidate_ref": "urcand1_docdemo", "evidence_hash": "…",
    "decision": "accept", "actor_id": "usr_…", "reason": "cleared our CI",
    "surface": "pull_request", "decided_at": "2026-09-15T00:00:00.000Z"
  }'

Expected state. With a valid signature: { "status": "accepted", "callback_id", "learning_use": "audit" }. A re-sent identical callback resolves to "duplicate" — a visible no-op, never a second recording — because the callback_id is derived from the callback's own identity. On the dashboard, the same decision is the candidate card's Approve rollout / Refuse form: it, too, requires a reason and records an audited receipt, and it, too, never presses the rollout itself.

6. Live confirmation and adoption — what you will see

This is the step the UI does not yet drive with a button, so here is the honest description of the state, not a screenshot.

When a rollout is started (FW10), a confirmation opens against a frozen baseline serving profile — the exact effective-config version at open time. The candidate is served to a treated arm; a fixed lever_holdout slice is served the frozen baseline so the contrast is randomized and live. The lift ledger renders only as an interval with its sample size, gated by an A/A false-positive check; superiority (the lower interval bound above zero) is required before any improvement is claimed, and the preregistered cost constraint must pass. No lever ever reads holdout traffic.

Adoption is a lease, not a switch. An adopted lever carries an expiry and reverts to the frozen baseline unless someone renews it — "if nobody renews, the lever reverts to the frozen baseline at expiry" is the designed default, so nothing runs unwatched forever. On the Flywheel screen the adopted lever appears under Adoptions (leases) with its granted/expiry dates and Renew lease / Roll back to baseline actions.

On the candidate card itself you will see the three evidence blocks from §"The three evidence kinds" side by side — each with its own authority label and verdict — a single progression headline (Collecting → Pattern found → Ready to test → Passed application checks → In live comparison → Confirmed / No improvement / Inconclusive → Adopted / Rolled back), the live lift interval when one exists, and the rollout gate rendered as a report (unlocked, or blocked with each named reason). The Supporting cases link opens the candidate's own run trace — the five-section what-happened / what-was-measured / result / inputs-and-outputs / lineage view, with bodies shown as content-hash pointers and gated by consent and retention.

7. Rollback and the epoch law

Rollback is available two ways, both shipped:

• The lease rollback — Roll back to baseline on an active lease, with a mandatory reason and an audited receipt, returns the call type to the frozen baseline.
• Automatic invalidation — an application deployment or rollback that changes the bound baseline invalidates an open confirmation (@ultrarouter/core#confirmationInvalidation) rather than silently crediting an unrelated change as lift. A changed head is stale_head, a changed config is app_config_changed, a changed epoch is epoch_changed, and a changed candidate is candidate_changed. Evidence from two measurement epochs is never pooled — the code throws rather than combine them.

So a merged PR is never, by itself, "measured improvement", and a deploy under a running confirmation never quietly becomes the result.

Troubleshooting

What this proves — and what it does not

Proves. You can carry a screened candidate into your own application CI, ask the server whether a controlled rollout may start, and get back a pressed: false report whose every refusal is named and whose missing evidence fails closed — with the offline / application-CI / live evidence kinds kept type-distinct at every step, a human decision recorded as an audit-only receipt, and adoption modeled as a revocable lease with rollback. All of it makes zero paid model calls.

Does not prove. It does not prove the candidate is better. Only a live confirmation with a superior interval can do that, and this build does not yet expose the press that opens confirmation or the one-click adopt — so no lift figure, and no business-outcome label, is claimed here. A passing CI check, a merged PR and a model-generated correction are, by construction, none of those things. When FW10 wires the press, the same candidate on the same workspace will move through In live comparison to one of Confirmed / No improvement / Inconclusive, with the interval, sample and inspectable failures behind it.

Next steps

• Declare what "it worked" means for your outputs first — see Declare what "it worked" means.
• Plan a straight model swap instead — see Plan a model migration.
• Register the provider keys a live confirmation dispatches through — see providers & BYOK.