AI Models

The Model You Pick Is a Price Decision

Frontier benchmarks move monthly. Your invoice does not. Choose on cost per successful task, not on the top row of a leaderboard.

3 min readPublished November 18, 2026
Share on X

Every few weeks a new model takes the top row of a leaderboard, and every few weeks somebody rebuilds their product around it. Then the invoice arrives. The score moved three points; the cost per conversation moved by a factor of six.

A model is not a trophy. It is a line item with a latency, a context limit and a failure mode attached. The useful question is never which model is best, it is which model is cheapest to run at the quality my users will accept.

Three numbers decide the bill

  • Output tokens cost several times more than input tokens on almost every provider, and a chatty model writes more of them than you asked for.
  • Cached input is the only discount most providers offer at scale. A stable system prompt that is re-sent on every turn is the difference between a cache hit and a full-price prefill.
  • Retries are invisible in a price list and visible on the invoice. A model that fails structured output one time in five costs 20% more than its sticker price.

Price lists change constantly, so do not memorise them. Write the arithmetic into your own code instead: a small `costOf(usage, prices)` function with the prices in config. When a provider changes a number you change one object, and your dashboard tells you the truth the next day.

// Prices live in config, never in prose.
const PRICES = { input: 3, output: 15, cacheRead: 0.3 }; // USD per 1M tokens

export function costOf({ input = 0, output = 0, cacheRead = 0 }) {
  return (input * PRICES.input + output * PRICES.output + cacheRead * PRICES.cacheRead) / 1e6;
}

Measure cost per successful task

Tokens per request is the wrong denominator. A cheap model that needs three attempts to produce valid JSON is more expensive than a dearer one that gets it right first time — and slower for the user, which is a second bill. Track cost per completed task, where a task is the thing your product promises: a resolved ticket, a generated invoice, a merged pull request.

The best model is the cheapest one that still passes your own eval, with enough headroom that a bad day does not become an outage.

Context length is a budget, not a feature

A 200k-token window is not an invitation to paste the repository. Attention cost grows with the sequence, prefill latency grows with it too, and everything you add dilutes the instruction you actually care about. Two thirds of the tokens in a typical agent prompt are files the model did not need.

Treat the window like RAM. Load what the current step needs, drop what it does not, and make the retrieval step explain why each file was included. If your agent cannot say why a file is in context, it is probably not needed.

A decision rule that survives a new release

  • Start with a mid-tier model and a 30-task eval built from your own logs.
  • Escalate to a frontier model only for tasks that fail the eval, and route those explicitly.
  • Keep a cheap, deterministic fallback for every route, so a provider incident degrades quality instead of stopping the product.
  • Re-run the eval when prices or models change, not when a tweet says so.
  • AI Models
  • Cost
  • Architecture
KI

Kadirul IslamVerified author

Founder, A.K Solution

Follow on X

Founder of A.K Solution. Full-stack engineer building AI agent tools like KiCode, JSON form engines like ki-forms, and high-scale SEO systems. Obsessed with shipping fast, zero-bloat software that lasts.

More from Kadirul Islam

A Benchmark Is a Map, Not the Territory

A model that wins a reasoning test can still fail to fill in four JSON fields. Build the eval your product actually needs.

2 min readAI Models

Open Weights Moved the Buy-versus-Build Line

The argument about self-hosting is not ideology, it is arithmetic. Find your volume before you pick a side.

2 min readAI Models

The Platform Replaced Half Your Utility Library

:has(), container queries, popover, light-dark(). The primitives that used to justify a dependency now ship in the browser.

3 min readCSS