Comparisons

Different question, different tool.

rightmodeler is an offline audit that measures cheaper models against outputs you already accepted. It is not a router, a gateway, or an eval platform, but it reads their traces and rides their rails. Each page draws the line.

The rails

They move your live requests. rightmodeler rides them as test benches and hands back a decision, not a proxy.

OpenRouterOpenRouter routes live traffic and, on Auto, picks models from community spend. rightmodeler replays your traces against outputs you accepted. They compose.

LiteLLMLiteLLM runs your request path: one API, budgets, fallbacks. rightmodeler replays your traces offline to measure which model belongs at each step. Keep both.

Vercel AI GatewayVercel AI Gateway serves and meters your live model calls. rightmodeler replays your traces to test cheaper models first. Complement: same credits, two jobs.

The graders

They score outputs against bars you define. rightmodeler asks which steps overpay to clear them, on the same traces.

BraintrustBraintrust holds your agent to evals you author. rightmodeler checks cheaper models against the outputs you shipped, step by step. They compose.

LangSmithLangSmith watches, scores, and runs agents in production. rightmodeler reads its exported run trees and decides which model each step needs. Different job.

promptfoopromptfoo tests what you ship next against cases you author. rightmodeler audits what you shipped against accepted outputs. Different stages of the same loop.

The routers

They predict the right model per request, at runtime. rightmodeler measures it per step, at release time.

Not DiamondNot Diamond predicts the best model per request, live. rightmodeler measures which model belongs at each step, offline, from your traces. Different clocks.

MartianMartian is a research lab with a name-the-model gateway; its router is legacy. rightmodeler replays traces offline to pick each step's model. Different jobs.