Integration · OpenTelemetry
rightmodeler + OpenTelemetry
If your stack emits gen_ai.* spans, the inputs and accepted outputs already exist. rightmodeler reads OTel GenAI telemetry from any OTLP pipeline, no bespoke integration required, measures cheaper candidates against those outputs, and reports the evidence.
TL;DR
rightmodeler reads OTLP span exports through a dedicated OTel GenAI adapter: any pipeline that emits gen_ai.* attributes, whatever framework or language produced it, folds into the same per-step schema. Spans are recognized by gen_ai.operation.name, grouped into trajectories by trace ID and ordered by start time, and read for model, messages, output, system instructions, and token usage. Each step is then replayed against cheaper candidates and scored against the output you already shipped.
How it works with OpenTelemetry
Land OTLP spans on disk
Dump OTLP trace data as JSON: a collector file exporter, your backend's span export, or spans captured from a test run all work. Check that content capture is on so gen_ai.input.messages and gen_ai.output.messages carry real payloads, then save the file anywhere on disk.
Autodetected by the gen_ai.* attributes
The adapter recognizes the format when a span's attributes carry gen_ai.operation.name, and requires a model in gen_ai.request.model or gen_ai.response.model. Spans sharing a trace ID become one trajectory, ordered by start time, and every span in a multi-span trajectory has to carry one. Messages come from gen_ai.input.messages, the answer from gen_ai.output.messages, the system prompt from gen_ai.system_instructions, and usage from gen_ai.usage.input_tokens and gen_ai.usage.output_tokens.
Cheaper candidates, judged against shipped output
Each step is replayed against cheaper candidates through the OpenAI-compatible endpoint you name with --base-url and --api-key-env. A configured external evaluator scores the outputs when it is reachable; otherwise the built-in judge runs two temperature-zero calls with the reference and candidate positions swapped, choosing a judge from a family that is neither the candidate's nor the reference's. Verdicts are confidence lower bounds kept separate per evaluator kind.
A per-step verdict you act on
The run writes report.md and report.json with a verdict per step family and streams JSONL events as it works. Thin evidence produces an abstention with a named reason. When a family clears every release gate, rightmodeler apply opens a draft pull request that changes model identifiers and nothing else, watch reconciles it against CI, and rollback restores the pre-apply files, refusing unless every restored digest matches.
Setup
Nothing to install: npx runs the published CLI. Hand it a span export. Nothing connects to your collector or tracing backend; the audit runs offline on files you already have.
# preview the pipeline against your spans, without spending
# then run it against your replay endpoint
# or let your coding agent drive it
What rightmodeler reads from OpenTelemetry
Every run is folded into one per-step schema, the same one the replay and the judge run on.
autodetected by: gen_ai.operation.name · gen_ai.request.model
Use cases
One adapter for everything that speaks OTel
Framework SDKs, auto-instrumentation agents, hand-rolled tracing: if it emits gen_ai.* semconv spans, it lands in the same audit through one adapter. Adopting a new framework means no new integration work; the telemetry contract does the carrying.
Right-size a polyglot stack
Services in different languages that share nothing but the semantic conventions fold into the same per-step schema, so a single audit covers the Python agent and the TypeScript sidecar alike, and shows which step families in each genuinely earn a frontier model.
Name your own step families
The adapter reads a rightmodeler.family attribute first and falls back to gen_ai.prompt.name, so if your instrumentation already labels prompts you get families that match how you think about the pipeline rather than a flat list of spans.
The honest part
- It reads OTLP span files on disk, JSON or JSONL. It does not connect to your collector, subscribe to a live OTLP endpoint, or scrape your tracing backend.
- Replay needs real payloads. Many GenAI instrumentations leave message-content capture off by default, and a span without gen_ai.input.messages or gen_ai.output.messages is dropped with a recorded reason rather than replayed.
- A trajectory of several spans has to carry a start time on every span, or those spans are dropped: without one the order of a multi-step trajectory cannot be recovered, and a guessed order would change what the judge sees.
- OTel spans usually carry no cost, so historical spend is not derived from them. Candidate calls are priced from what your replay endpoint reports, and marked as an estimate when the provider returns none.
- Evidence minimums are per evaluator kind: at least 10 assessed executions, at least two distinct step IDs, and at least five distinct trajectories. Below any of those the family abstains with a named reason instead of recommending.
- It is a report, not a collector processor or a gateway. Your OTel pipeline keeps exporting spans; rightmodeler never sits in the request path.
Frequently asked questions
How do I export OpenTelemetry spans for rightmodeler?
Land OTLP trace data on disk as JSON or JSONL: the OTel Collector's file exporter works, as does a span export from your backend or a capture from a local test run. Pass the file or folder with --traces; the format is autodetected from the gen_ai.* attributes.
My framework has no dedicated adapter. Am I stuck?
No, that is what the OTel path is for. Anything that emits GenAI semantic-convention spans is readable through the same adapter, whatever SDK or language produced it. If you can get gen_ai.* attributes into an OTLP export, you can run the audit.
Why are my spans ingested but not replayed?
Most likely the message content is missing. GenAI instrumentations often record model names and token counts by default but treat message content as opt-in; without gen_ai.input.messages there is nothing to replay a candidate on. Enable content capture in your instrumentation and re-export.
Does rightmodeler replace my tracing backend?
No. The backend watches production; rightmodeler runs offline on exported spans and answers a question dashboards leave open, whether a cheaper model would have produced equivalent output on the steps you actually ran. Keep both; they read the same telemetry.
Does my span data leave my machine?
Replays send step inputs to candidate models through the endpoint you configured, under your own key, the same shape of call your service already makes to a provider. There is no rightmodeler server and no account; nothing else touches your spans.
Will it always find savings?
No, and by design. A family abstains when the evidence is thin, when the excluded fraction runs over the ceiling, or when no candidate clears the quality floor again on held-out cases. A tool that always finds a swap is not measuring anything.
Run the audit on your own traces
The CLI runs from npx, nothing to install; your OpenTelemetry traces do the rest.