Controller live·4,812 rollouts gated this weekv1.4

Every prompt change ships behind a quality gate.

Repath sits between your app and your model provider. It splits traffic, scores every response with a judge model, and pulls the candidate back to zero the moment quality drops — before a single user notices.

Start free Try the controller

Free while you evaluate·No card·One base_url to integrate

rollout / support-triagev4 → candidate

livenext tick 24s

Judge quality score

gpt-4o-mini · rolling 200-sample mean

baselinecandidate

Quality

0.920.68

P95 latency

812ms

Error rate

0.00%

Samples judged

14,208

Controller decisions

ROLLBACK50% → 0%
quality 0.68 < 0.70now

HOLD50% → 50%
quality 0.72, drifting down2m

ADVANCE10% → 50%
quality 0.93 ≥ 0.9018m

ADVANCE0% → 10%
gate passed · 1,204 samples44m

STARTrollout created
baseline gpt-4o pinned1h

Written to postgres with the metrics that caused it

A candidate degrading at constant latency and zero errors — caught and pulled at 0.68

Gating production traffic at

logo

01 — The silent failure mode

Nothing errors. Nothing alerts. The answers just get worse.

97.6 → 2.4%

Accuracy fell off a cliff

A 2023 Stanford study measured GPT-4's accuracy on one coding task dropping from 97.6% to 2.4% inside a month. The API returned 200 the entire time.

0 errors

Your dashboards stay green

Status codes, latency and error rates are blind to quality. A regression that halves usefulness looks identical to a clean deploy.

34 days

Found by customers, not by you

A subtle prompt edit degraded responses for over a month before anyone tied the support tickets back to the deploy that caused them.

Feature flags tell you the code deployed. Repath tells you whether it worked.

02 — How it works

One line in. A gate around every change.

Point your existing SDK at Repath. Routing, recording, judging and deciding all happen off the request path — the Rust gateway adds under 2ms, and nothing in your hot path waits on an evaluation.

pythonthe entire integration

# before

client = OpenAI(api_key="sk-…")

# after

client = OpenAI(api_key="sk-…", base_url="https://api.tryrepath.com/v1")

YOUR APP · base_url = api.tryrepath.com/v1

REPATH GATEWAY · rust / axum · <2ms

TRAFFIC ROUTER
90% baseline · 10% candidate

REQUEST RECORDER
async · never blocks

OPENAI · ANTHROPIC · GEMINI · OPENROUTER
baseline and candidate

response returns immediately
evaluation happens beside it

REDIS STREAM · eval-queue

PYTHON EVALUATOR
checks + gpt-4o-mini judge

POSTGRES 16 · scores

RUST CONTROLLER · every 30s
≥ 0.90 advance / < 0.70 rollback

Drop-in replacement

Change a base URL. Keep the provider SDK you already use.

Traffic splitting

Send a slice of real requests to the new prompt or model. Both sides scored on identical criteria.

Automatic rollback

Below threshold, candidate weight goes to zero on the next tick. No pager, no human.

03 — Live demo

Move the score. Watch the controller decide.

This runs the real decision rule: advance at 0.90, hold between, rollback under 0.70. Drag the score and step the controller through the ladder.

Candidate quality score

0.93

0.000.70 rollback0.90 advance1.00

Traffic to candidate

step 1 of 3

90%

baseline gpt-4ocandidate v4

Decision log

tick #12

ADVANCE0% → 10%
gate passed · 1,204 samples

04 — What it does

Four primitives. Everything else is configuration.

4.1

Canary deployments for prompts

Ladder a new version 10% → 50% → 100%. Every step carries its own gate, and the controller — not a human on a Friday — decides whether it passes.

rollout.yaml

steps:
  - weight: 10
    gate:
      quality_score: ">= 0.9"
  - weight: 100

4.2

A judge model scores every response

Describe what good looks like in plain English. Repath scores each response with gpt-4o-mini plus your programmatic checks. No metric schemas to design.

judge.yaml

judge_prompt: |
  Score this response 0-1.
  Criteria: accuracy, clarity,
  relevance to the query.

4.3

Rollback in under 500ms

When the rolling score crosses your threshold, candidate weight is zero before the next request is routed. Not eventually — on that tick.

controller.yaml

controller:
  check_interval: 30s
  rollback_threshold: 0.7
  action: instant

4.4

Every decision is auditable

Advance, hold, rollback, promote — each stored with the exact metrics that triggered it. Enough to answer "why did this change?" months later.

decisions · postgres

{
  "action": "rollback",
  "reason": "quality 0.68 < 0.70",
  "previous_weight": 50,
  "new_weight": 0
}

05 — Compared

Flags ship it. Observability watches it. Repath decides.

Capability	Repath	LaunchDarkly	LiteLLM	Langfuse
Canary deployments for prompts		Enterprise	—	—
LLM quality evaluation		—	—	View only
Automatic rollback on quality		Enterprise	—	—
Open source		—
Self-hostable		—
Price for startups	Free	$100K+/yr	Free	Free

06 — Self-host

Running locally in 60 seconds.

The hosted product is the same code. BSL 1.1, converting to Apache 2.0 after four years — self-host it forever if you’d rather.

Clone and configure

git clone github.com/repathhq/repath
cd repath && cp .env.example .env

Start the stack

docker compose up
# gateway :8080 · console :3000

Create a rollout

repath rollout create \
  -f examples/demo-canary.yaml

Read the full documentation →

Ship the next prompt change without holding your breath.

Connect a base URL, define one gate, and let the controller hold the line. Free while you evaluate.

Start free Talk to an engineer