AI Agents

Your Agent Needs a Router, Not a Single Model

One model call per turn is simple until it is slow, rate-limited, or expensive. Routing is the fix and it is not hard.

2 min readPublished June 12, 2026
Share on X

The first version of my coding agent called one model. Every prompt, one provider, one key, one model. It worked, and then it was slow on a large refactor, and the failure mode was that there was nothing to fall back to.

The fix was not a better model. It was a router.

Why one model per turn does not scale

Three different kinds of request arrive in the same stream: a file search, a one-line edit, and an architecture argument that needs a long context. They have wildly different cost and latency profiles and no reason to share a configuration.

  • A cheap, fast model handles tool calls, file reads, and classification.
  • A strong model handles planning and anything that changes several files at once.
  • A fallback exists for when the strong one is rate-limited, which is always.

Routing on purpose, not on vibes

The routing rule is the whole design. Mine is intentionally boring: quota first, then prompt shape, then a probe of what the provider is actually offering right now.

1. M1  quota      - is there budget left for this turn?
2. M2  probe      - what is the upstream provider healthy right now?
3. M3  health     - which configured routes are actually up?
4. M4  policy     - pick the strongest route that passed the above
5. M5  fallback   - on failure, demote and try the next viable route

The part that matters is the outcome hook. A fallback that only triggers on an exception misses the case that actually hurts: a provider returning 200 with degraded output, or a stream that dies halfway through. The router has to learn from the result of every call, not just from whether it threw.

What this buys

Predictable cost on the common path, real headroom when the strong model is unavailable, and — the part I care about most — a single place where a provider degradation is visible instead of being absorbed silently across the codebase.

  • AI
  • Architecture
  • Cost
KI

Kadirul IslamVerified author

Founder, A.K Solution

Follow on X

Founder of A.K Solution. Full-stack engineer building AI agent tools like KiCode, JSON form engines like ki-forms, and high-scale SEO systems. Obsessed with shipping fast, zero-bloat software that lasts.

More from Kadirul Islam

A Context Window Is Not Memory

Re-reading the same tokens on every turn is not the same as remembering them, and the difference is your bill.

2 min readAI