Applied AI · build note Production tool

What you can check without an LLM, you don't ask an LLM.

A router I run in production. It sends every task to the cheapest engine that can pass a deterministic check, and only spends the expensive model on the work that actually needs judgment.

Independent build · applied AI Python · Groq · Claude

Route a task yourself ↓

The console

Pick a job. Watch it find the cheapest path.

The fast model drafts. Plain code checks it for free. The expensive model is called only when a check fails. This runs the same four checks that live in the repo, on your click.

llm_router · route(prompt, checks=…) deterministic gate

prompt →

1 Draft · Groq

—

→

2 Checks · Python

deterministic · $0

→

3 Resolve

—

0tasks routed

0resolved by Groq · $0

0escalated to Claude

—without the paid model

The policy

Three stages, in order of cost.

Ordering matters: the free steps run first, and the paid one is a last resort with no free pass of its own.

STAGE 01

Draft

Groq · ~0.5s · $0

A small fast model (gpt-oss-120b) writes, classifies, extracts, or scores. Key rotation and a local gateway fallback mean it rarely stalls.

STAGE 02

Judge

Python · $0

Plain code decides if the draft is good: banned phrases, length, valid JSON, a required decision line. Anything verifiable this way never reaches a model.

STAGE 03

Escalate — only if needed

Claude Opus · paid

On a failed check, a sensitive task, or a QA sample, Claude fixes it and runs the same checks. If it's rate-limited, the best draft ships flagged unreviewed rather than dropping the pipeline.

Inputs, outputs, tooling

The spec, and what it saves at scale.

Input	A prompt + a set of deterministic checks. The caller declares what "good" means as code.
Output	Finished text and which engine produced it, logged per task. A classify job takes raw text in and returns validated JSON out.
Draft engine	Groq — gpt-oss-120b. Six rotating keys, OmniRoute gateway as fallback when the free tier runs dry.
Judge	Deterministic Python. Reusable checks: words, banned phrases, JSON, decision line. Cost zero.
Escalation	Claude Opus via the CLI on subscription — never a raw API key. Its output clears the same checks.
Telemetry	JSON per task and engine → python llm_router.py stats. If you can't measure the split, you can't claim it.
Failure mode	Degrades honestly. No paid model available → ship the best draft, marked unreviewed and counted.

Modeled at scale

Drag the first-pass rate to see the split on a 1,000-task mix.

750paid-model calls avoided per 1,000 tasks

First-pass rate — 75%

Groq · $0 750 250 Claude · paid

A model of the split, not a usage figure. Real per-task counts live in the repo's telemetry; this slider just shows how the ordering pays off as volume grows.

The proof it ships with

Every decision is counted.

The tool's own report. The paid column is the one I want small.

$ python llm_router.py stats

🚦 LLM ROUTER — load split

task                     groq  gateway  claude  fails  % saved
----------------------------------------------------------------------
draft_outreach        12       1      5      0       72%
classify_reply        19       0      4      0       83%
extract_json           8       1      2      0       82%
score_lead             9       0      6      1       60%
----------------------------------------------------------------------
TOTAL                 48       2      17              75%

→ 50 calls resolved without Claude.
→ of those, 2 were rescued by the gateway when Groq couldn't.

On the numbers. The console above shows the report's real format and a representative run. The interactive checks are the genuine article, ported to run in your browser. I'm happy to walk through the source: it's ~390 lines, one file, and the decision I'm proudest of is the one where a rate-limited paid model degrades to an honest draft instead of a dead pipeline.