The rightmodeler blog

Notes on running agents on the right models.

How we measure candidates against accepted outputs, watch every layer, and turn model changes into code review.

Guide · LLM costs

LLM cost optimization tools: which approach fits your workload?

Six ways to cut an LLM bill, sorted by the problem each solves: what it needs, the evidence to demand, and how they stack.

· 12 min read

Engineering guide

How an AI gateway can invalidate your model evaluation.

Your evaluation gets a 200 and a score, but the answer may come from a fallback, an alias, a cache or a rewritten request. Four failure modes, the check that catches each, and the fix.

· 12 min read

Comparison · Observability

Langfuse vs Helicone: tracing, evaluation, gateways, and cost analysis.

Two open-source answers built from opposite ends: what each vendor’s docs say about tracing, evaluation, gateways, caching and cost, where both companies stand, and when to run both.

· 11 min read

Checklist · Gateways

Bifrost vs LiteLLM: a practical evaluation checklist.

Both front many providers with one OpenAI-compatible API. The differences come later: where keys live, what the catalog says about price, what the logs keep, and what changes the model.

· 11 min read

Research · Token economics

Nobody can predict the bill.

You approve the run and nobody knows the price, including the agent about to spend it. What a new paper on agent token costs actually measured, and what it changes about picking models.

· 7 min read

The vision · Part one

The bill nobody can read.

An agent feature ships, the invoice arrives as one number, and nobody can say which layer spent it. Part one of our vision: giving agents the instruments every other system already has.

· 5 min read

The vision · Part two

The Tuesday problem.

The group chat asks: are we switching? And everyone goes quiet. Part two of our vision: why model migrations should arrive as pull requests with the evidence attached.

· 6 min read

Founding story

We were picking models by vibes. Then the bill arrived.

We shipped agentic systems for years while guessing which model belonged at which step. rightmodeler is the tool we kept rebuilding by hand to stop guessing, made properly.

· 5 min read