AI Models

Open Weights Moved the Buy-versus-Build Line

The argument about self-hosting is not ideology, it is arithmetic. Find your volume before you pick a side.

2 min readPublished October 27, 2026
Share on X

The argument about open-weight models is usually framed as ideology. It is arithmetic. For a fixed workload there is a volume below which an API is cheaper and a volume above which a GPU is. The useful work is finding your volume, not picking a side.

What self-hosting actually costs

The model weights are free. Everything around them is not: a GPU that bills by the hour whether or not a request arrives, a serving stack (vLLM or llama.cpp, both excellent and both something you now operate), quantisation choices that trade quality for memory, monitoring, an upgrade path, and the person who gets paged at 2am when the container runs out of VRAM.

  • Idle cost — a dedicated GPU is a fixed monthly bill. An API is zero when nobody is using the product.
  • Throughput cost — at high utilisation, self-hosted tokens are dramatically cheaper per million.
  • People cost — the most expensive line item, and the one missing from every comparison blog post.

A small product with spiky traffic almost never wins on self-hosting. A product with a steady queue, a privacy constraint, or a fine-tune that must not leave the building often does.

The three cases where open weights win outright

  • Data residency — the input cannot legally cross a border. No API contract fixes that; a GPU in your own rack does.
  • A narrow, repeated task — classification, extraction, embedding. A 7B model fine-tuned on 5,000 labelled examples can beat a general frontier model on that one task, at a fraction of the price.
  • Latency floor — a local model on the same machine as the application removes a network hop and a rate limit from the critical path.

The hybrid that usually wins

Run the cheap, high-volume, well-defined work on a local model and send the rare hard case to an API. The router decides by task type, not by confidence score — a model's self-reported confidence is not a calibrated probability, and building a control flow on it is how you get a system that escalates everything or nothing.

const route = (task) =>
  task.kind === "extract" || task.kind === "classify"
    ? localModel          // steady volume, narrow task
    : frontierModel;      // rare, open-ended, worth the price

Buy the frontier. Own the commodity. Route between them by task, not by vibes.

One more thing worth writing down: open weights are a continuity guarantee. If a provider changes its terms or retires a model, a self-hosted checkpoint keeps working. That is not a cost saving; it is insurance, and it is worth paying for when the product depends on it.

  • AI Models
  • Infrastructure
  • Cost
KI

Kadirul IslamVerified author

Founder, A.K Solution

Follow on X

Founder of A.K Solution. Full-stack engineer building AI agent tools like KiCode, JSON form engines like ki-forms, and high-scale SEO systems. Obsessed with shipping fast, zero-bloat software that lasts.

More from Kadirul Islam

The Model You Pick Is a Price Decision

Frontier benchmarks move monthly. Your invoice does not. Choose on cost per successful task, not on the top row of a leaderboard.

3 min readAI Models

A Benchmark Is a Map, Not the Territory

A model that wins a reasoning test can still fail to fill in four JSON fields. Build the eval your product actually needs.

2 min readAI Models

The Platform Replaced Half Your Utility Library

:has(), container queries, popover, light-dark(). The primitives that used to justify a dependency now ship in the browser.

3 min readCSS