Source: https://www.rightmodeler.com/integrations/github # rightmodeler + GitHub When an audit proves a cheaper model is safe for a step, the change reaches GitHub as a draft pull request your team reviews like any other. It swaps model identifiers and nothing else, carries its evidence in the body, and asks the people who own the code to look. Source control · draft pull requests Official site: https://github.com rightmodeler works with GitHub through apply, watch and rollback. apply opens a draft pull request that changes only model identifiers, with an evidence table in the body and reviewers taken from CODEOWNERS, then git blame. watch makes one pass over that pull request per run: it answers review questions with the stored evidence, marks families for re-proof when a reviewer requests changes or the base branch changes a swapped file, and closes the pull request only when the same check fails again on the same commit. rollback opens a draft that restores a merged swap. None of them merges anything. ## How it works with GitHub ### 01 · Check: Every gate runs before GitHub changes apply needs a completed init run that recommends a swap, a local HEAD on a branch at the evidence revision with that branch on GitHub at the same commit, and no uncommitted edits to the files the swap touches. The proposed diff is linted: a change to anything but model identifiers refuses with diff_lint_failed, and a formatter that rewrites other content refuses with formatter_blocked. --dry-run runs every check, reads GitHub, writes nothing, and prints the branch, title, body, files and reviewers. ### 02 · Open: A draft that swaps model identifiers only apply creates the branch and commits the change through GitHub's API, then opens the pull request as a draft. The branch uses the prefix your recent branches use most, else rightmodeler/, and the title follows your commit style: perf(models): swap summarize in a repository that uses conventional commits, Swap summarize models otherwise. The body is your pull request template followed by a Rightmodeler evidence table with the revision, the corpus version, and per family the decision, evaluators, cascade status, worst-case bound, models, cost per case, latency, caps and case IDs. Case IDs are SHA-256 digests of the replayed cases, never prompts. With --code-graph, a Graphify code context section for the swapped call sites follows the table; it never changes the swap or its reviewers. ### 03 · Review: Code owners are requested by apply, not by GitHub GitHub does not request code owners on a draft pull request, so apply makes every request itself. Owners of each swapped file come from the first CODEOWNERS file it finds in .github/, the root or docs/, where the last matching rule wins. A file with no matching rule falls back to its three most recent git blame authors, matched to GitHub users by commit email. At most five users and five teams are requested, never the pull request author, and if GitHub rejects the batch each reviewer is requested alone. ### 04 · Watch: One pass per run, and never a merge Each watch run reconciles one pull request under a lock kept in the store, so overlapping runs cannot act twice. It answers each human review or comment once with the stored evidence for the families involved, marks families for re-proof when a reviewer requests changes or the base branch changes a swapped file, and comments once on a failing check or status. If a check with the same name fails again on a new run of the same commit, it closes the pull request. A merge ends watching, and a close without merge is recorded as a rejection that apply honors until there is new evidence. ### 05 · Roll back: A merged swap comes back the same way rollback, given the merged pull request's number, opens a draft that restores the pre-apply contents of every file the swap changed. It refuses unless the original pull request merged and those files still match their recorded post-apply state, and a rerun returns the same rollback pull request instead of opening another. ## Setup Run these from the repository's root once an init run recommends a swap. Pass the name of the variable holding the token, never the token itself, and add --github-repo when the folder name differs from the repository name on GitHub. ```bash # a token with the repo scope, such as the one gh auth token prints export GITHUB_TOKEN="$(gh auth token)" ``` ```bash # preview the branch, title, body, files and reviewers without writing npx rightmodeler apply --owner $OWNER --github-token-env GITHUB_TOKEN --dry-run ``` ```bash # open the draft pull request npx rightmodeler apply --owner $OWNER --github-token-env GITHUB_TOKEN ``` ```bash # reconcile it; run again on a schedule or after review activity npx rightmodeler watch --owner $OWNER --pr $PR --github-token-env GITHUB_TOKEN ``` ```bash # or let your coding agent drive it npx skills add elm-os/rightmodeler --skill rightmodeler ``` ## Use cases ### Review a model swap like any other change The swap reaches your team as a draft pull request whose diff touches model identifiers and nothing else, so your CI, linters and branch protections check it like every other change. Reviewers read the evidence table in the body instead of a benchmark someone pasted into chat. ### Ask the evidence a question in the thread A reviewer can ask which cases back the swap in an ordinary comment, and the next watch pass replies once with the stored evidence for that family: decision, evaluator, passes, trials, pass rate, worst-case bound and case IDs. The reply is built from the ledger, not written by a model. ### Send a swap back for proof A review that requests changes, or a base-branch commit that edits a swapped file, marks the affected families for re-proof and says so in the thread, so stale evidence does not ride along into a merge. ### Undo a swap without overwriting later work If a merged swap misbehaves, rollback opens a draft that restores the pre-apply files. It refuses when those files have changed since the swap merged, rather than overwriting someone else's edit. ## The honest part - rightmodeler never merges. The GitHub client it ships has no merge call: apply and rollback open drafts, watch comments and can close a pull request after a repeated check failure, and the merge stays with a person. - watch acts only when it runs. Each run is a pass over one pull request, so schedule it or trigger it from repository events; nothing listens to GitHub in between. - Re-proof is a mark, not a rerun. watch records which families need new evidence and says so in the thread, and the new evidence comes from your next pipeline run. - A fine-grained personal access token cannot read check runs, because GitHub offers those tokens no check-run permission. watch still reconciles reviews, comments, commit statuses, merges and base-branch changes, and prints the github_checks_unavailable warning. - The blame fallback needs full history and a commit email GitHub can match to a user. An owner it cannot match is not requested, team reviewers need a repository owned by an organization, and every reviewer needs access to the repository. - Verified live in a disposable repository: apply with GitHub Actions' GITHUB_TOKEN opened drafts authored by github-actions[bot] and requested the CODEOWNERS owner, and watch answered a reviewer's question once, both with that token and with the owner's own token. Nothing was merged. GitHub App tokens and fine-grained tokens are covered by tests against a GitHub stub that models their permissions, not by a live run, and GitHub Enterprise Server has not been tested live. ## FAQ ### Which GitHub tokens work? Pass the name of the variable that holds the token with --github-token-env, never the token itself. A GitHub App installation token, a classic personal access token with the repo scope, or the token gh auth token prints runs apply and watch fully. A fine-grained personal access token runs apply fully and watch without check runs. Inside GitHub Actions, GITHUB_TOKEN runs both when the job grants contents: write, pull-requests: write and statuses: read, plus checks: read for check runs, and the repository allows GitHub Actions to create pull requests. ### Can rightmodeler merge the pull request? No. The GitHub client it ships has no merge call. apply and rollback open drafts, watch comments and closes a pull request only after a repeated check failure, and a person reviews and merges. ### Who gets asked to review? The owners of each swapped file, from the first of .github/CODEOWNERS, CODEOWNERS and docs/CODEOWNERS that exists, with the last matching rule winning. A file no rule covers falls back to its three most recent git blame authors, matched to GitHub users by commit email, so fetch full history in CI. At most five users and five teams are requested, and never the pull request author. ### What happens when a reviewer asks a question? The next watch pass replies once, in the thread, with the stored evidence for the families the comment names, else the families in the file it comments on, else every family in the pull request. A review that requests changes instead marks those families for re-proof, and the new evidence comes from the next pipeline run. ### How do I undo a swap after it merged? Run rightmodeler rollback with --owner, --pr set to the merged pull request's number and --github-token-env. It opens a draft pull request that restores the pre-apply contents of the files the swap changed, and it refuses unless the original pull request merged and those files still match their recorded post-apply state. ### Does it work with GitHub Enterprise Server? It is built for version 3.21 or newer. Pass the server's API URL with --github-base-url, such as https://github.example.com/api/v3. Older servers answer every call with HTTP 400, because the CLI sends REST API version 2026-03-10, which GitHub Enterprise Server supports from 3.21. No Enterprise Server has been tested live yet; the live runs used github.com.