Now open·7-day free trial, no card

Every prompt change ships behind a quality gate.

Repath sits between your app and your model provider. It splits traffic, scores every response with a judge model, and pulls the candidate back to zero the moment quality drops — before a single user notices.

Start free Try the controller

Free while you evaluate·No card·Works with your existing OpenAI SDK

rollout / support-triagev4 → candidate

livenext tick 24s

Judge quality score

LLM judge · rolling mean

baselinecandidate

Quality

0.920.68

P95 latency

812ms

Error rate

0.00%

Samples judged

14,208

Controller decisions

ROLLBACK50% → 0%
quality 0.68 < 0.70now

HOLD50% → 50%
quality 0.72, drifting down2m

ADVANCE10% → 50%
quality 0.93 ≥ 0.9018m

ADVANCE0% → 10%
gate passed · 1,204 samples44m

STARTrollout created
baseline gpt-4o pinned1h

Written to postgres with the metrics that caused it

A candidate degrading at constant latency and zero errors — caught and pulled at 0.68

01 — The silent failure mode

Nothing errors. Nothing alerts. The answers just get worse.

97.6 → 2.4%

Same model name, different model

A 2023 Stanford and Berkeley study measured GPT-4 identifying prime numbers at 97.6% in March and 2.4% in June. Same endpoint, same model name, a 200 on every call.

0 errors

Your dashboards stay green

Status codes, latency and error rates are blind to quality. A regression that halves usefulness looks identical to a clean deploy.

via tickets

Found by customers, not by you

Without a quality signal on live traffic, a worse prompt is found the slow way — in support tickets, long after the deploy that caused it.

Feature flags tell you the code deployed. Repath tells you whether it worked.

02 — How it works

One line in. A gate around every change.

Point your existing SDK at Repath. Routing, recording, judging and deciding all happen off the request path — the Rust gateway adds under 2ms, and nothing in your hot path waits on an evaluation.

pythonthe entire integration

# before

client = OpenAI(api_key="sk-…")

# after

client = OpenAI(api_key="sk-…",

base_url="https://api.tryrepath.com/v1",

default_headers={"X-Repath-Key": "rp_live_…"})

YOUR APP · base_url = api.tryrepath.com/v1

REPATH GATEWAY · rust / axum · <2ms

TRAFFIC ROUTER
90% baseline · 10% candidate

REQUEST RECORDER
async · never blocks

OPENAI · ANTHROPIC · GEMINI · OPENROUTER · VERCEL
baseline and candidate

response returns immediately
evaluation happens beside it

REDIS STREAM · eval-queue

PYTHON EVALUATOR
checks + LLM judge

POSTGRES 16 · scores

RUST CONTROLLER · every 30s
≥ 0.90 advance / < 0.70 rollback

Drop-in replacement

Change a base URL. Keep the provider SDK you already use.

Traffic splitting

Send a slice of real requests to the new prompt or model. Both sides scored on identical criteria.

Automatic rollback

Below threshold, candidate weight goes to zero on the next tick. No pager, no human.

03 — Live demo

Move the score. Watch the controller decide.

This runs the real decision rule: advance at 0.90, hold between, rollback under 0.70. Drag the score and step the controller through the ladder.

Candidate quality score

0.93

0.000.70 rollback0.90 advance1.00

Traffic to candidate

step 1 of 3

90%

baseline gpt-4ocandidate v4

Decision log

tick #12

ADVANCE0% → 10%
gate passed · 1,204 samples

04 — What it does

Four primitives. Everything else is configuration.

4.1

Canary deployments for prompts

Ladder a new version 10% → 50% → 100%. Every step carries its own gate, and the controller — not a human on a Friday — decides whether it passes.

rollout.yaml

steps:
  - weight: 10
    gate:
      quality_score: ">= 0.9"
  - weight: 100

4.2

A judge model scores every response

Describe what good looks like in plain English. Repath scores responses with a fast judge model plus your programmatic checks. No metric schemas to design.

rollout.yaml

evaluation:
  - type: llm_judge
    criteria:
      - name: helpfulness
        prompt: "Does this give specific,
          actionable help?"

4.3

Rollback without a human

The controller checks every rollout every 30 seconds. When quality crosses your threshold, the candidate goes to zero on that check, and the gateway stops routing to it within seconds.

rollout.yaml

rollback:
  trigger:
    quality_score: "< 0.7"
    error_rate: "> 0.1"
  action: instant

4.4

Every decision is auditable

Advance, hold, rollback, promote — each stored with the exact metrics that triggered it. Enough to answer "why did this change?" months later.

decisions · postgres

{
  "action": "rollback",
  "reason": "quality 0.68 < 0.70",
  "previous_weight": 50,
  "new_weight": 0
}

05 — Compared

Flags ship it. Observability watches it. Repath decides.

Capability	Repath	Feature flags	LLM gateway	LLM observability
Split traffic between prompt or model versions			Weighted routing	—
Score live responses for quality		Add-on	—
Advance or roll back on quality, automatically		Add-on	—	—
Every rollout decision logged with its scores		—	—	—
Drop-in for OpenAI-compatible clients		—		Via SDK
Self-hostable	Source-available	Varies	Varies	Varies

06 — Self-host

Running locally in 60 seconds.

The hosted product is the same code. BSL 1.1, converting to Apache 2.0 after four years — self-host it forever if you’d rather.

Clone and configure

git clone github.com/repathhq/repath
cd repath && cp .env.example .env

Start the stack

docker compose up
# gateway :8080 · console :3000

Create a rollout

repath rollout create \
  -f examples/demo-canary.yaml

Read the full documentation →

Ship the next prompt change without holding your breath.

Connect a base URL, define one gate, and let the controller hold the line. Free while you evaluate.

Start free Talk to an engineer