For two years the answer was easy. You called a closed API, because the closed models were simply better and the gap was wide enough that nothing else mattered. A bank with strict data rules paid for a private deployment of a closed model and made peace with it. The open models were a hobbyist's option, a year or more behind, fine for a side project, not for production.
That answer expired sometime in 2025. The open-weight families caught up far enough that, for most workloads, capability no longer decides. What is left is a genuine engineering decision with real money and real risk on both sides, made one workload at a time. This guide is the honest version of it: what open weight actually means, how far the gap really closed, and the per-workload framework for choosing. The short version: most enterprises end up running both, on purpose.
What "open weight" actually means
Start by killing a confusion, because it changes what you are agreeing to. Open weight is not open source.
An open-weight model is one whose trained parameters, the weights, are published for download. You can pull them, run them on your own hardware, and fine-tune them. Llama, DeepSeek, Qwen, GLM, Mistral, and Gemma are all open weight. That is a real freedom, and also the limit of what most of them give you.
Open source, in the sense the term means for software, would additionally require the training code and the training data. Almost no major model releases those. DeepSeek published its weights and a detailed technical report, but not the dataset, and there has been public guessing ever since about what it trained on. The Open Source Initiative has been blunt that "open weights" is often marketed as more open than it is: weights without data and code are a downloadable artifact, not a reproducible one. You can run and adapt the model. You cannot rebuild it or fully audit it.
For an enterprise the practical reading is simple. Open weight buys three concrete things: the right to run the model on infrastructure you control, the right to fine-tune it on your own data, and freedom from a vendor's pricing and lifecycle decisions. It does not buy a full provenance trail of the training data. The three you get are usually the ones that matter, but know which is which before a compliance conversation.
How far the gap actually closed
The capability story is the part that changed most. Be precise about it, not triumphant.
Benchmark numbers move fast, so treat what follows as a dated snapshot, May 2026. On general benchmarks the open-weight tier is now a short distance behind the closed frontier, not a generation behind. On BenchLM's composite ranking, the strongest open-weight model, DeepSeek V4 Pro, scores 87, against Gemini 3.1 Pro at 93 and GPT-5.4 Pro at 92. A six-point spread, where two years ago it was a gulf. The narrower picture is even better for the open side. On SWE-Bench Pro, the harder coding benchmark, GLM-5.1 and Kimi K2.6 land around 58 percent, level with a mid-tier closed model like GPT-5.5 even as the very top closed coders still pull ahead. The current DeepSeek generation is among the strongest models anywhere on hard math. For classification, extraction, summarization, retrieval-augmented question answering, and most coding, a good open-weight model in 2026 is not a compromise. It does the job.
The remaining closed-model edge is real but specific. It shows up in long-horizon agentic reasoning: a task that runs twenty, thirty, or more dependent steps, where an early mistake has to be caught and corrected rather than assumed away. On those workloads the frontier closed models still hold a measurable lead, keeping coherence and backtracking more reliably across a long chain. They also lead on the consistency of safety fine-tuning. So the gap did not vanish. It narrowed to a specific, hard, valuable corner. If your workload lives there, capability still decides for you. If it does not, and most workloads do not, you have a real choice on other grounds.
The decision, one workload at a time
There is no enterprise-wide answer to open versus closed, only a per-workload one. These are the dimensions that produce it.
Data sovereignty and privacy
This dimension can settle the question on its own, before cost or capability enters. Some data legally cannot leave infrastructure you control: protected health information, certain categories of financial data, defense and government work, and personal data under strict residency rules. A closed API, by definition, sends your tokens to someone else's servers, often in another country. Enterprise tiers and zero-retention agreements reduce the exposure, but for the strictest workloads they do not eliminate the legal problem of data crossing a boundary.
Self-hosting is the only architecture that keeps data fully inside your perimeter, and self-hosting requires weights you can download. This is the clearest single case for open weight. Meta publishes a dedicated deployment guide for running Llama inside a HIPAA boundary because the demand is concrete. The regulatory pressure is rising too: EU data residency rules are driving a measurable move toward self-hosting, and the EU AI Act becomes fully applicable on 2 August 2026. For a regulated workload the question is often not "which is cheaper" but "which is even allowed," and the answer is the one you can run yourself.
Cost at scale
Cost is the dimension most teams get backwards, reasoning from a single number instead of a volume curve.
At low and moderate volume, the closed API is cheaper, and it is not close. Someone else bought the GPUs, amortizes idle time across thousands of customers, and charges per token. Self-hosting means renting or buying GPUs that cost the same whether saturated or idle, plus the engineering to run them.
The crossover comes at high, steady volume, and the break-even line sits in a wide band depending on which closed model you compare against. Against a frontier-tier model, self-hosting an open-weight equivalent can pay off in the low millions of tokens per day. Against a cheap small closed model the break-even shifts into the tens of millions, because the API price you are trying to beat is already low. One concrete comparison: at 50 million tokens a day, a small closed model via API runs about 2,250 dollars a month, while the same workload self-hosted on rented GPUs runs over 5,000. Volume alone did not make self-hosting win there; the workload also has to keep the GPUs busy.
Two honest caveats keep this from being a clean win. First, the GPU bill is not the real bill. Self-hosting costs roughly three to five times the raw hardware price once you add deployment, load testing, autoscaling, observability, and the standing slice of an engineer's time to keep it healthy. Second, idle capacity is your problem now: a workload heavy in business hours and quiet at night leaves paid-for GPUs unused most of the week. Self-hosting rewards high and steady. It punishes spiky. For why per-token prices fell so far in the first place, see the inference cost collapse.
Control and stability
Closed models get deprecated on the vendor's schedule, not yours. Providers retire model versions roughly every 12 to 18 months, and on managed platforms the upgrade can be forced: when two GPT-4o snapshots retired in March 2026, Azure deployments were auto-upgraded to a newer model. For a chatbot that is an annoyance. For a regulated agent validated against a specific model, a silent swap is a compliance event, because the thing in production is no longer the thing that was approved.
An open-weight model you have downloaded does not retire. The weights sit in your storage and behave the same next year as today. You upgrade when your testing says to, not when a vendor's calendar does. You also escape pricing changes and the small but nonzero risk of losing API access. The cost of that stability is that you own the maintenance and the security patching. You do not get stability and zero operational burden both.
Customization
Prompting and retrieval get you a long way, and for most teams they are enough. When they are not, the next step is fine-tuning, and full fine-tuning needs the weights. Closed providers offer hosted fine-tuning, but it is bounded: you tune within the lane the vendor allows, and cannot inspect or move the result. With open weights you can do full fine-tuning, run adapter methods like LoRA, distill the model smaller, quantize it for cheaper serving, and keep the artifact. A workload that depends on deep behavioral or domain shaping beyond what hosted tuning reaches needs open weights. Otherwise this dimension is neutral.
Latency, offline, and edge
A closed API needs a network round trip to a data center. For most applications that is fine. For some it is not: a factory floor, a point-of-sale device, a vehicle, a secure facility with no outside connection, or any workload where every millisecond counts. A self-hosted model can run next to the application, or on the device itself, with no round trip. If a workload has to run offline, at the edge, or under a hard latency budget, open weight is the only option that fits. This connects directly to small language models, which usually fill those constrained places.
What the closed models still do best
A fair guide names the other side plainly. Closed models keep three advantages worth paying for.
The first is frontier reasoning on the hardest problems. For long-horizon agentic work the closed flagships are still ahead, and if that is your workload the premium is fair.
The second is zero infrastructure burden. With an API there is nothing to provision, patch, scale, or monitor, and a team can ship the day they get a key. For a team that would rather spend its engineering on the product than on a GPU fleet, that is worth a great deal.
The third is managed safety and updates: the provider runs the safety fine-tuning, handles the security work, and ships improvements you inherit for free. With a self-hosted model that is your job. For a large share of workloads the closed call is the right one, which is exactly why this is a decision and not a slogan.
How to actually decide
Walk each significant workload through the dimensions in order and stop at the first decisive one.
Does the data legally have to stay inside your perimeter? If yes, self-host an open-weight model and the question is settled. If no, continue. Does the workload need frontier-grade long-horizon reasoning? If yes, a closed flagship is worth the premium. If no, continue. Is the volume high and steady enough to clear the self-hosting break-even, with the full three-to-five-times engineering cost included? If yes, open weight likely wins on economics; if the volume is low or spiky, the API is cheaper. Finally, does the workload need deep customization, offline operation, or version stability the API cannot promise? Each points to open weight. If no dimension is decisive, default to the closed API, because the operational simplicity is real and free.
Run that for every workload and you will not get one answer. You will get a portfolio. That is what enterprises are doing: F5's 2026 research found organizations running or evaluating an average of seven models, with 78 percent operating some inference themselves. The mature pattern is a hybrid: self-hosted open weights for the sensitive, high-volume, latency-sensitive, or offline work, closed APIs for the hardest reasoning and the bursty long tail, and a routing layer deciding per request. The new operational skill is not picking the one model. It is running the portfolio.
Where this is heading
Two forces are pulling in opposite directions, and the tension is the story of the next few years.
The open-weight tier keeps closing the gap. The release cadence is faster than the closed labs', the cost structure is brutal, and an analysis from Berkeley's management review frames the trajectory as classic disruption: open models started cheaper and are climbing the capability curve fast, the same shape as Linux against proprietary operating systems. Strategy reinforces it. Europe's push for sovereign AI treats open weights as strategic infrastructure, not a budget option, because a model you can self-host is one no foreign provider can switch off.
Pulling the other way, the closed labs are not standing still. The hardest long-horizon agentic reasoning is exactly where they keep investing, so a frontier edge on the most demanding work is likely to persist. The realistic forecast is not that one side wins. It is that the open-weight tier becomes the default for the broad middle of enterprise workloads, while the closed frontier holds the top end and the convenience case. The decision in this guide does not go away. It gets sharper, and it stays per-workload.
Council summary
This guide argues that open weight versus closed is no longer a capability question for most enterprise work: the open-weight tier now sits a handful of benchmark points behind the closed frontier, not a generation behind. What remains is an engineering decision, settled per workload along five dimensions, data sovereignty, cost at scale, control and stability, customization, and latency or offline need, walked in order until one is decisive. No single answer survives that exercise. Run it across a portfolio and you land on a hybrid: self-hosted open weights for sensitive, high-volume, or edge work, closed APIs for the hardest reasoning and the bursty tail. The takeaway is to stop hunting for the one model and build the routing, observability, and governance muscle to run several at once.
Comments