Manifesto
Prove it. Don't guess.
A model downgrade is a real decision. It deserves evidence, not a vibe.
The default is over-provisioned.
The biggest model on every step feels safe. Then the bill compounds, call by call, and nobody can say which step earned it.
model usage · Juneillustrative
route · gpt-5.6$1,180
extract · gpt-5.6$940
summarize · gpt-5.6$2,310
judge · gpt-5.6$860
rerank · gpt-5.6$1,040
embed · gpt-5.6$620
month to date$5,290 and climbing
Evidence beats vibes.
Leaderboards are not your workload. Every candidate is replayed on your own traces and judged against what you shipped.
A category, not a feature.
Evidence-backed model downgrading: detect, prove, fix. A report you run today, pull requests next, continuous with Crucible.
Review the evidence
illustrativerightmodeler-agent
swap: summarize step to gpt-5.4-mini
Q 0.94 · 85% cheaper · 214 traces replayed
Replays, scores, and confidence attached. The merge stays yours.
CloseMerge
It can say no.
Weak evidence means abstain.
Your traces are the benchmark.
Judged against what you shipped, never a leaderboard.
Nothing swaps on its own.
Risks flagged, evidence attached, merge yours.
FAQ
- What is evidence-backed model downgrading?
- Moving a step to a cheaper model only after proving, on your own traces, that quality holds against the output you already shipped. The decision is backed by evidence: replays, scores, and a confidence level, not a benchmark or a hunch.
- How is this different from observability?
- Observability shows you what happened. It doesn't replay your steps through cheaper models, judge the results, or change anything. rightmodeler proves a specific swap is safe and applies it in your repo: detect, prove, fix.
- Is it safe to downgrade automatically?
- rightmodeler never swaps on its own. It abstains when the evidence is weak, flags cascade risk, and leaves the final call, and the repo edit, to you.