Learning controls

A call type that learns from its own verified episodes keeps a history of versions — each one a small, immutable set of lessons distilled from episodes your application reported as passed or failed. The learning controls decide which version your agent is served, and let you see and steer every step.

A newly learned version is a candidate. Your promotion policy decides when it replaces the active version. With evaluate, the candidate serves only its randomly assigned share of eligible episodes during the comparison; the other arm uses the current version. Promotion makes the candidate the active version for subsequent traffic:

Negative or inconclusive evaluations keep the active version. Nothing on this page claims that a learned version improves your agent — an evaluation is exactly how you find out, on your own traffic, with an interval you can read.

Prerequisites

• Learning available for your workspace. Learning can be switched off for a workspace on a deployment (it is off by default on a new deployment). Then every learning route answers 404 not_found, exactly like an unknown route, and the dashboard shows no Learning pages; routing and compression keep working. Ask your ultrarouter operator to enable learning for the workspace.
• Workspace Learning opt-in on. As the owner, choose Allow learning on the call type's dashboard Learning page or in the Learn onboarding flow. This grants both corrections consent to store episode content and knowledge consent to use what it learns. Corrections consent alone is insufficient. The equivalent request is PUT /v1/workspaces/{id}/self-learning with {"enabled":true} from a signed-in owner session; a bearer API key cannot grant these permissions.
• A reflection model your workspace owns — a model served by one of your custom providers (for example a local subscription gateway) or a provider you stored your own key for. Learning never calls a platform-keyed model and has no paid fallback. Your workspace's billing funding mode must allow calls on your own keys (byok).
• An owner or admin session, or an API key with config:write. Reading needs config:read (or any membership).

Turn learning on

curl -sS -X POST "$UR_BASE_URL/v1/call-types/$CALL_TYPE_ID/learning/enable" \
  -H "Authorization: Bearer $UR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "reflection_model": "local/claude-sonnet" }'

The response carries the resulting status. The promotion policy starts from these defaults — change any of them with a promotion block, before an evaluation starts:

Once an evaluation starts, its rule is frozen with it (and hashed): changing the policy affects the next evaluation only.

Choose the learner recipe

recipe picks how a reflection learns. Each field is optional; an omitted field keeps the original learner's value:

curl -sS -X POST "$UR_BASE_URL/v1/call-types/$CALL_TYPE_ID/learning/enable" \
  -H "Authorization: Bearer $UR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "reflection_model": "local/claude-sonnet", "recipe": { "validator": "elv2", "generation": "atomic", "observations": "elo2" } }'

The recipe is stored on the call type's episode-memory setting, so GET …/learning (recipe) and GET …/episode-memory (setting.lessons.recipe) always report the same one. Enabling again without recipe keeps the current one; sending the all-default recipe returns to the original learner. A new recipe applies from the next reflection; each version records the recipe hash it was built with.

What elv2 admits

elv2 gives each proposed lesson two separate verdicts:

• Evidence support. The lesson must cite known cases, and every tool it names must be offered in, or called by, at least one cited case. Otherwise it is needs_stronger_evidence (named_tool_not_in_cited_evidence). An uncited case never grounds a tool.
• Value handling. Every literal is classified by its role in everything the reflection saw, cited or not. A copied identifier, user value or answer is never admitted. A quoted message that appears only in tool outputs, never in a user turn, a tool argument or an answer, and holds no task-specific value is accepted (for example 'Booking not found'). A constant is admitted only when a tool's schema or documentation states it and the lesson names that tool somewhere in its text. A constant documented for one tool never vouches for a lesson that names only another.

A tool name that reads as an identifier (get_user_tickets, displayCarStatus) names that tool wherever it appears. A tool named by an ordinary word (add, find) is named only when the lesson writes it as a call (add(), with the api. prefix, in backticks, or as the whole value of an assignment or call argument. Elsewhere the word is ordinary language: in "find the id with api.get_user_tickets()", only get_user_tickets is named.

The admission rules carry a revision number that is folded into the recipe hash, so a version built under a newer revision reports a different epr2_ hash.

Document text-protocol tools

Agents that call tools in code (api.fillFuelTank(...)) instead of sending native tools[] give the learner no schema to read constants from. Supply the specs once per call type, as OpenAI-style function definitions:

curl -sS -X PUT "$UR_BASE_URL/v1/call-types/$CALL_TYPE_ID/learning/tool-documentation" \
  -H "Authorization: Bearer $UR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "tools": [ { "type": "function", "function": { "name": "fillFuelTank", "description": "The tank holds at most 50 liters.", "parameters": { "type": "object", "properties": { "fuelAmount": { "type": "number" } } } } } ] }'

• At most 256 specs with unique names, and a body of at most 1 MiB.
• Each spec is stored as a bounded summary: its name, a tsf1_ schema fingerprint, its parameter names and the constants it documents (enum / const / default values and numbers or quoted tokens from its description). The description prose itself is never stored.
• PUT replaces the stored set. Sending the same set again returns changed: false, and an empty tools list clears it. GET returns what is stored.
• Changing it needs config:write (or an owner/admin session); reading needs config:read. Every change is audited with counts, digests and fingerprints only.
• Revoking corrections consent erases it with the rest of the learning data.

The documentation only affects elv2 and only from the next reflection. It never grounds a tool for evidence support: that still needs a cited case.

How an evaluation works

New evaluations use lpr2. While enrollment is open, each eligible episode is assigned the candidate or active version by a stable hash of its episode key. The assignment is recorded before the selected lessons are read, and continuing steps keep that assignment. A retrieval fallback serves no lessons on that request; it does not remove an enrolled episode's outcome.

Enrollment closes at the configured episode-count threshold. The selected episode IDs are frozen before their outcomes are inspected. New episodes then use the active version, while the selected episodes finish under their original assignments. Only whole-episode outcomes your application reports through a teaching channel count, such as a verifier result, resolved ticket or accepted correction. Self-ratings and helpfulness labels never decide a promotion.

Selected episodes with no verdict stay unknown; they are not dropped or replaced. The comparison waits for their outcomes within its original deadline. Missing outcomes at that deadline keep the incumbent active. This comparison concerns recorded, enrolled episodes; it does not establish a guarantee for all incoming traffic or future tasks.

Once the selected outcomes are complete, the gateway computes the one-sided Newcombe interval on candidate − active success and decides once:

• promoted — the lower bound is above 0 and the improvement is at least min_effect_pp;
• rejected — the candidate is significantly worse;
• retained / inconclusive — anything else; the active version stays.

Historical lpr1 evaluations retain their original verified-only rule and result. An unfinished evaluation using that retired rule is stopped conservatively; it is not silently reinterpreted as lpr2.

curl -sS "$UR_BASE_URL/v1/call-types/$CALL_TYPE_ID/learning/evaluations" \
  -H "Authorization: Bearer $UR_API_KEY"

A newer candidate that arrives while an evaluation runs waits for it; the newest waiting candidate is evaluated next and any older waiting one is marked superseded.

Read the status

curl -sS "$UR_BASE_URL/v1/call-types/$CALL_TYPE_ID/learning" \
  -H "Authorization: Bearer $UR_API_KEY"

The evidence block separates what arrived (recorded, verified, unknown) from what the learner has reflected over (consumed) and what is in the active version (incorporated_active); backlog is verified but not yet reflected over. reflection.next says whether the next scan will reflect and why not. usage reports reflection tokens separately from cash.

outcome_receipts counts the outcomes your application reported through POST /v1/requests/{id}/outcomes for this call type's requests, by state: pending (accepted, the request is not recorded yet), applied, rejected, failed, expired (the request was never recorded in time), and episode_pending (applied, waiting for its episode to be recorded). A receipt counts once its request or its episode has been recorded for the call type.

Steer it

Versions are immutable, so a rollback serves exactly the lessons that version had. Every control is recorded in your workspace audit log. Revoking corrections consent erases the versions, their history, the evaluations and the controls.

Troubleshooting