LLM Router: Use the Expensive Model Only Where It Earns Its Keep

Premium models (Opus, GPT-4o) cost $30-60 per million tokens. Haiku and gpt-4o-mini — $0.50-2. A 30x gap. And yet 80% of real requests are classification, rephrasing, field extraction, short FAQ answers. At scale, you're paying 30x for the same answer. Analogy: imagine a company where a senior engineer does everything — from bug reports to architecture. Junior tasks get billed at senior rates. A team with seniority levels is an order of magnitude cheaper and faster — because the junior isn't stuck in senior meetings. An LLM router is the same idea: the right level for the right task.

To route smartly, you need to know what kind of request this is. The trap: if the classifier is too smart (= expensive), the savings are eaten by the classifier itself. If too simple, it misroutes, and requests fly into the wrong model. The right balance — a cheap model + structured output + a fallback policy. Haiku with JSON output classifies in <200ms for <$0.0001 per request. That's a tiny routing tax that pays itself back on the first correct dispatch to Haiku instead of Opus. Signals to classify on: (1) task type — summarize, code, reason, classify; (2) input length — a cheap model wastes money on long context; (3) expected output length; (4) needs tools/function calls; (5) domain expertise level — legal advice and 'rephrase this' are different planets.

The cascade pattern: start with the cheapest model, check quality, escalate only on failure. Haiku goes first, Sonnet picks up harder cases, Opus kicks in on genuinely tough ones. Most traffic settles on Haiku — and so does the budget. Why this beats parallel voting (three models answering at once): in voting you always pay for all three. In a cascade you only pay for the expensive path on failures. The difference for a typical 70/20/10 tier load: voting costs the sum of all three on every request; cascade costs Haiku + 30% Sonnet + 10% Opus. A 3-5x gap. Key nuance: when the cascade escalates, pass the junior model's answer as a hint to the senior. Opus sees where Haiku stumbled — that's both task context and an explicit anti-example. The final answer quality beats what Opus would produce solving from scratch.

cascade(запрос):
  ответ_1 = Haiku(запрос)
  если quality_gate(ответ_1) == "ok":
    вернуть ответ_1

  ответ_2 = Sonnet(запрос, hint=ответ_1)  # использует контекст
  если quality_gate(ответ_2) == "ok":
    вернуть ответ_2

  вернуть Opus(запрос, hints=[ответ_1, ответ_2])
  # Opus видит, где младшие ошиблись — больше шансов не повторить

A quality gate is the function that decides whether the cheap model actually did the job. Without a gate you returned a bad answer and saved $0.01 — that's not optimization, that's degrading your product for pennies. Three types of gate, from simple to complex. First — programmatic: schema check, regex, length, JSON structure. Costs zero, catches gross failures. Good for structured output: classification, extraction, form-filling. Second — self-critique: the cheap model evaluates its own answer against a stricter prompt. 'Is this answer complete? Any contradictions?' Costs one more Haiku call, catches semantic failures — answer is syntactically correct but factually empty. Third — retrieval grounding: check that the answer is backed by real sources. For RAG pipelines, must-have. Cites a source not in the retrieved set — hallucinated, escalate. The health metric for your gate is fallback rate — share of escalated requests. <20% — saving well. 20-60% — normal. >60% — junior model too low, start the cascade at Sonnet. A bad gate is worse than no router: Haiku quality at Opus prices.

Without metrics the story is always the same: in two weeks someone says 'quality is down', management panics, and you roll back to a single premium. Router buried. With metrics, you see exactly which request class is suffering and fix it surgically without touching the rest. The five metrics below are the minimum. Track them per request class, not as an average: an overall average will hide a problem in 10% of traffic that breaks a key scenario. And always keep a baseline — the same traffic on a single premium — otherwise there is nothing to compare against.