Self-hosted models

If you run your own model behind an OpenAI-compatible server (vLLM, TGI, Ollama, LM Studio, …), register it as a custom provider and route to it like any other.

1. Expose an OpenAI-compatible endpoint

For example, vLLM serves the OpenAI API:

python -m vllm.entrypoints.openai.api_server \
  --model meta-llama/Llama-3.3-70B-Instruct \
  --served-model-name my-llama --port 8000

That gives you http://your-host:8000/v1/chat/completions.

2. Register the provider

curl -sS -X POST "$UR_BASE_URL/v1/workspaces/$WS_ID/providers" \
  -H "Cookie: better-auth.session_token=$UR_SESSION" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My vLLM",
    "base_url": "https://your-host:8000/v1",
    "auth": { "header": "Authorization", "value_template": "Bearer {{secret}}" },
    "model_ids": ["my-llama"],
    "pricing": { "input": 0, "output": 0 }
  }'

Then store the upstream token for it (envelope-encrypted, never returned):

curl -sS -X POST "$UR_BASE_URL/v1/workspaces/$WS_ID/provider-keys" \
  -H "Cookie: better-auth.session_token=$UR_SESSION" \
  -H "Content-Type: application/json" \
  -d '{ "provider_id": "'"$PROVIDER_ID"'", "secret": "your-upstream-token" }'

pricing is in integer micro-USD per 1M tokens (input, output, optional cached). Declare it — the gateway only routes to an offer whose cost it can compute. A provider registered without pricing is refused before dispatch with the model_unpriced error (HTTP 403), because its calls would otherwise be recorded as $0 where spend caps, budgets and savings cannot see them. A self-hosted model you do not pay per token for declares 0, as above. That is a price you stated, and those requests record a stated $0.

The base URL is SSRF-validated on save (and on every update). The native model id is sent upstream verbatim.

Your workspace pays this endpoint with your own token, so choose BYOK funding once (a new workspace starts on prepaid UR credits, which never pay a provider you run):

curl -sS -X POST "$UR_BASE_URL/v1/workspaces/$WS_ID/billing/funding" \
  -H "Cookie: better-auth.session_token=$UR_SESSION" \
  -H "Content-Type: application/json" \
  -d '{ "mode": "byok", "allowPlatformFallback": false, "requestKey": "byok-choice-0001" }'

Without it the request is refused with HTTP 403 permission, naming this route. See Providers & BYOK. Then route to it:

{ "model": "my-llama", "messages": [{ "role": "user", "content": "hi" }] }

3. Declare what the endpoint supports

Add a capabilities object to tell the router what your endpoint can do. The known flags are tools, json_schema, vision, streaming, and embeddings. Omit it and every flag defaults to true (the assumed OpenAI-compatible set) — existing providers are unchanged.

curl -sS -X PATCH "$UR_BASE_URL/v1/workspaces/$WS_ID/providers/$PROVIDER_ID" \
  -H "Cookie: better-auth.session_token=$UR_SESSION" \
  -H "Content-Type: application/json" \
  -d '{ "capabilities": { "json_schema": false } }'

A flag set to false makes the router refuse a request that needs it before dispatch, with the named capability_unsupported error — never a silent downgrade of your response_format and never a wasted upstream call. For example, an endpoint that does not implement structured outputs should declare "json_schema": false, so a response_format: { "type": "json_schema" } request is rejected up front instead of failing at the provider. The same flags appear on GET /v1/models, so discovery agrees with dispatch. Change them any time from the API or the dashboard /providers form.

Security notes

• HTTPS only. Plain http:// + loopback is allowed only when the gateway runs with UR_ALLOW_INSECURE_CUSTOM_PROVIDERS=true (dev/e2e only).
• Private, link-local, and cloud-metadata IP ranges are rejected; redirects are re-validated (max 2).
• The upstream secret is envelope-encrypted and never returned by the API.

The dashboard /providers screen offers a guided form with a live "test connection" and a real invoke.