Review and curate what the flywheel learns

Our promise is automated learning: the flywheel turns your everyday usage into lessons without asking you to label anything. The review queue is the human pipe beside that promise — a place you may use to grade your own drafts and help curate what is learned, but never have to. Nothing here is a prerequisite for the flywheel to work.

The queue lists the drafts for one call type — the model outputs captured from your imported history and your live passthrough traffic — and lets a human give each one a verdict. That verdict is a declared correction: your judgement on our output, never an oracle of correctness and never a quality score.

Where it is

In the dashboard under Flywheel → Review queue (/flywheel/review). The landing page lists each call type that has drafts to review with its progress; open one to reach that call type's queue at /flywheel/review/<call-type-id>.

What each item shows

• The draft — your model's output at the exact version a reviewer saw, as a bounded excerpt. Captured-passthrough bodies are read through the one audited, tenant-isolated CEE accessor (every read is recorded in your audit log); import bodies are the intact content-addressed blobs the importer stored. The draft is the model's answer text whatever the provider shape — OpenAI chat completions, Anthropic Messages, OpenAI Responses or a reconstructed stream — not the raw envelope. When the answer is JSON it is shown pretty-printed (indented, syntax-tinted, with collapsible objects and arrays) and you can flip between Pretty and Raw; plain text is shown as-is.
• The source material — the request that produced the draft, when it is present, with the same pretty/raw rendering.
• The AI reviewer's suggestion, when one exists — its verdict, its reason codes and the repaired text a machine verifier proposed. If no suggestion exists, the item simply shows none; we never fabricate one.

Bodies are shown only under corrections consent. With consent off you can still approve or reject, but drafts, edits and suggestions are neither shown nor stored — turn consent on from the Flywheel screen to edit and to accept suggestions.

The four verbs

Each item offers four actions, keyboard-complete with a / e / s / r on the active item:

• Approve as-is (a) — records correction:accepted_unchanged. The draft was good; your acceptance is the lesson.
• Edit (e) — opens an inline editor seeded with the full draft (loaded on demand, not the shortened excerpt), so you edit the whole body. If the draft is JSON your edit must stay valid JSON — you get a named inline error otherwise. Save to record correction:edited with a character-level diff (computed on the server against the full draft, never a value sent from your browser); your edited text is stored as the human-approved final.
• Accept the AI's suggestion (s) — one click makes the verifier's repaired text your declared final, recorded correction:edited and marked human-accepted (the accepted receipt id rides along), so provenance stays honest: a human accepted a machine suggestion — it is never relabelled as machine-approved. Shown only when a suggestion exists.
• Reject with a reason (r) — records correction:rejected with one closed reason code (off topic, inaccurate, tone, incomplete, formatting, policy, other). A rejection carries the code alone — never a free-text note in an outcome value.

Leave a feedback note (optional)

Any action can carry an optional free-text note — "what should change?", up to 2,000 characters. The note is content, so it is kept only under corrections consent: the text is stored in the encrypted blob store and linked to the draft's artifact and the correction you just recorded, while the outcome event itself still carries no text (at most a hash). With consent off the note box is disabled with a one-line reason, and nothing is saved. A stored note follows the same retention as your other correction bodies — it is swept with them and removed when you revoke consent. The flywheel may read your notes as evidence when it decides what to learn; a note is never a training label by itself.

Fix with AI

When you know what should change but don't want to type the whole revision yourself, write a feedback note and press Fix with AI. We ask your own model to revise the draft using your note (and your reject reason, if you picked one) and load the result into the editor for you to accept, change, or discard. If the draft is JSON, the revision keeps the same JSON shape.

This is a real model call, so it is your usage and your spend: it runs through the same metered gateway path as any other request — a call type named ur_review_assist, a request id you can find in your logs, and your workspace spend cap applies. The cost of the revision is shown next to the editor. If your workspace is at its monthly cap, Fix with AI pauses with a clear message until the cap resets or you raise it.

The AI's revision is never saved on its own — it only fills the editor. Nothing is recorded until you Save as final, which is a normal edit (a declared correction:edited with your final text). Fix with AI needs corrections consent on (we read the full draft to revise it); with consent off the button is disabled with a one-line reason.

What it does not do

• It never claims an edit, acceptance or rejection is an oracle of correctness.
• It never trains a model by itself: a declared correction is a lesson the flywheel may learn from under your governance, through the same gates as every other outcome; the holdouts are still skipped before any teaching reader reads a body.
• It is not a scoreboard and not required. Progress (n of N reviewed) is there if you want to work through a batch; leaving it at 0 changes nothing about the automated loop.

See also

• Capture your first reviewer correction — the webhook adapter that brings corrections in from a reviewer system you already run.
• Record outcomes — the outcome API the review queue records through.