Open weights now sit 1.7% behind the best closed model, which makes self-hosting a real option for regulated organisations. This framework prices both paths on measured traffic, names the four drivers that survive scrutiny, and sets out where the hybrid line belongs in your estate.
The trap most regulated organisations walk into: set the monthly API bill against hardware amortisation, size the cluster for peak from a pilot week, and call the result sovereignty, then run at 19% utilisation while marginal tokens feel free and the true cost per token sits at three to six times nameplate.
What the teams doing better do instead: start from the named clause or accreditation condition that hardware actually discharges, measure four full weeks of traffic before sizing anything, build the gateway before the cluster, and price cost per completed task with idle hours in the denominator.
A standard processing agreement with region pinning and zero-retention terms already covers a great deal of GDPR exposure. Hardware becomes necessary when something specific bites: national cloud rules naming approved operators, a defence classification level, sector limits on where regulated records may be processed, or a subprocessor list that can change on thirty days notice.
Pinned
A hosted version can be retired, deprecated or quietly adjusted while your prompts, thresholds and validation evidence stay where they were. For a marketing summary that is an inconvenience. For a credit decision tool a regulator has reviewed, the artefact you validated no longer exists and a decision from eight months ago cannot be reproduced.
It never leaves our network is a data flow statement, not a posture. Host it and you own the serving stack: model servers with a history of deserialisation and remote code execution bugs, weights pulled from public registries, GPU drivers, container runtimes, and a KV cache that leaks across tenants when isolation is careless.
Start from the named clause or accreditation condition, and write down what a contract genuinely cannot discharge. If no obligation survives that test, the case is a preference dressed as architecture and the hosted path is cheaper.
Peak-to-mean ratio over four full weeks, not a pilot week, which will oversize the cluster by a multiple. Then price cost per thousand completed tasks with idle hours in the denominator, because that is the number finance compares.
One internal API holding the policy: data classification in, model selection out, with per-team quotas and cost ceilings. One evaluation harness and one audit log across both backends, so moving a workload becomes a policy change instead of a rewrite.
Identifier, version, licence, source registry, artefact hash, quantisation, tokeniser, adapters and serving engine version. Verify the hash at load and log it with every batch, so the audit trail names the exact weights behind any response a regulator asks about.
It moves the risk rather than reducing it. You remove one egress path and take ownership of model servers with remote code execution history, public registry weights, GPU drivers, cache isolation between tenants, and all the input and output filtering the provider ran for you.
Usually not. Region pinning with zero-retention terms discharges much of the GDPR exposure. Hardware becomes necessary when an obligation names approved operators, sets a classification level, or forbids a subprocessor list
that can change on thirty days notice.
At sustained round-the-clock load. An eight-accelerator node costs near $200,000 a year before serving a token, and peak-sized clusters run at 15% to 30% utilisation, so real cost per token is three to six times nameplate. Measure four weeks of traffic first.
Long-horizon agentic work, complex multi-step tool use and difficult code generation. The capability gap still decides whether those workflows complete at all. Route them through the same gateway with redaction at the boundary, and keep one eval harness across both backends.
Licence terms are where that assumption breaks. They vary widely, some restrict commercial use above a threshold or derivative models, and most releases do not disclose training data. Record the provenance gap, the compensating controls and who accepted the residual risk in your technical file.
Drop your details and we'll send Self-Hosting Open Weights Transfers Responsibility, It Does Not Add Security straight to your inbox - no spam, unsubscribe anytime.
Bring your traffic profile and the obligation you are trying to discharge. Our engineers will cost both paths and tell you where the hybrid line sits.
Book an architecture review