Integrations

Works with the traces you already have.

No new SDK and nothing in your request path: rightmodeler reads the traces your agent already emits, measures cheaper candidates against accepted outputs, and reports the evidence, sample size, and abstentions.

Reads your traces

Autodetected and folded into one per-step schema.

Claude CodePoint rightmodeler at ~/.claude/projects: it reads your sessions as traces and measures cheaper models against the output you accepted, step by step.
Codex CLIRun rightmodeler on the Codex rollouts already on disk: it reads each session as a trace and judges cheaper models against the result you accepted.
LangSmithExport a LangSmith run tree: rightmodeler replays each step through cheaper models and measures every output against what you accepted.
the OpenAI SDKPoint rightmodeler at your OpenAI SDK request log: it replays every call through cheaper models and measures each output against what you accepted.
LangfuseExport Langfuse observations to disk and rightmodeler replays each step through cheaper models, judging every candidate against the output you already shipped.
BraintrustFetch Braintrust span rows: rightmodeler replays each step through cheaper models and measures every candidate against the output you accepted.
Arize PhoenixExport OpenInference spans from Phoenix: rightmodeler replays each step on cheaper models and measures every output against what you accepted.
OpenTelemetryPoint rightmodeler at OTLP span exports with gen_ai.* attributes. It replays each step on cheaper models and measures candidate agreement with accepted output.
Vercel AI SDKExport Vercel AI SDK spans: rightmodeler binds each functionId to its call sites, replays text calls on cheaper models, and judges them against your output.
HeliconeExport Helicone request rows: rightmodeler replays each recorded call against cheaper candidates and scores every one against the response you already shipped.
W&B WeaveExport W&B Weave calls: rightmodeler replays each recorded call against cheaper candidates and scores every one against the output you already shipped.

Grades the replays

Scorers you already trust grade each replay, in place of the built-in judge.

promptfooGrade rightmodeler replays with promptfoo: your assertions and named metrics decide which cheaper candidates pass, run locally on the real promptfoo CLI.

Runs the candidates

Where the candidate calls actually run.

OpenRouterPoint rightmodeler at OpenRouter with one base URL and one key: it shortlists cheaper models from the live catalog, replays your steps, and prices every call.
LiteLLMPoint rightmodeler at a LiteLLM proxy as its replay route: candidate calls and the judge run through the gateway you already operate, on your own keys.
PortkeyReplay and judge through the open-source Portkey gateway: request headers pick the upstream, and an answer from another model or a cache is never graded.
Agent Router (formerly Envoy AI Gateway)Replay and judge through Agent Router, formerly Envoy AI Gateway, and read its OpenInference spans as traces. Verified on v1.1.0 standalone and on Kubernetes.
BifrostReplay and judge through the open-source Bifrost gateway and read its request logs as traces. An answer from another model or from its cache is never graded.
Vercel AI GatewayPoint rightmodeler at the Vercel AI Gateway: it finds cheaper candidates in the live catalog, replays real steps, judges quality, and records reported costs.
Vercel SandboxRun rightmodeler's confirm stage in Vercel Sandbox: each case runs your app in a fresh microVM in your Vercel project, and the model key never enters it.

Ships the change

Where an approved swap becomes a change you review. rightmodeler never merges it for you.

GitHubrightmodeler apply opens a draft GitHub pull request that changes only model identifiers, carries the evidence and asks code owners to review. A person merges.
GitHub ActionsA tested GitHub Actions workflow runs rightmodeler every week, opens the draft pull request only when a person dispatches apply, and watches it on GITHUB_TOKEN.

Maps your code

Static context about the code around each swap, shown beside the evidence and never counted as evidence.

GraphifyBuild a code graph with open-source Graphify and rightmodeler lists the callers, tests, and owners of each tested call site in its report, never as evidence.

Coming soon

Not built yet. Each page says exactly what exists today and what does not.

ModaicNot built yet: a planned rightmodeler evaluator will let a judge in your Modaic account grade replays via Modaic's hosted API. What works now, what is planned.
RelatedHow it worksReduce LLM costsCompare alternativesGlossary