1. What exactly has been opened?
A recurring mistake in procurement is to treat “downloadable” as synonymous with “open source”. At least three states need to be distinguished: open access, open weights, and an AI system that meets an open-source definition.
Open access usually means that users can reach a model through a website or API, while the service provider does not deliver the weights. Users gain the ability to call the model, but cannot decide which machine runs it or restore it themselves after the endpoint closes. Open weights give users the trained parameters, making local inference and further modification possible, but training code, data documentation, evaluation tools, training scripts or redistribution rights may still be missing. Whether the result is an open-source system also depends on whether the components needed to modify it are available and whether the licence permits use, study, modification and sharing.
The Open Source Initiative's Open Source AI Definition 1.0 makes those freedoms explicit: users should be able to use the system, study how it works, modify it, and share the original or modified system. [2] This is a definition, not a legal provision automatically adopted in every jurisdiction, and it cannot replace review of the specific licence. Its practical value is to remind buyers that permission to call a model, availability of files and legal rights are not the same thing.
The Model Openness Framework approaches the question through components across the model lifecycle. It separates openness from completeness and lists elements including the model architecture, final checkpoint, technical report, evaluations, data card, and training and inference code. [3] It is not an authority that grants a legally binding “open source” certificate. It is a checking tool: a vague label should not conceal missing assets.
Specific releases show why this distinction matters. OpenAI released gpt-oss-120b and gpt-oss-20b as open weights under Apache 2.0. Its official README also provides inference examples, tools and deployment information, and says that with MXFP4 quantisation the 120B model can run on a single 80GB GPU while the 20B model can run within 16GB of memory. [4] The model card describes both as open-weight models and records their tokenizer and agentic tool-use design. [5] These materials support the conclusion that users receive a runnable and modifiable set of core assets. They do not establish that the training data, the chain of rights governing that data, or the complete training process has been disclosed.
Licensing can narrow or expand that control further. Section 1.b.i of the Llama 3.1 Community License sets requirements for attribution and model naming when the model is distributed. Section 2 imposes a separate licensing condition only at a specific threshold: on 23 July 2024, the release date of that version, the products or services offered by the licensee or its affiliates must have exceeded 700 million monthly active users in the preceding calendar month. An organisation meeting that condition must request a licence from Meta and cannot exercise the rights granted by the agreement unless Meta expressly authorises it. [6] This is more precise than the shorthand claim that “anyone above 700 million monthly users must apply”. The assessment has to return to the version, the legal entity and the applicable date.
DeepSeek-R1 presents a different case. The licence section of its official README says that the code repository and original model weights use the MIT licence. It also identifies the separate foundations of its distilled models: those based on Qwen derive from Apache 2.0 base models, while those based on Llama remain associated with the relevant Llama licence. [7] A repository's top-level licence therefore cannot be generalised to every downstream variant. Anyone migrating or redistributing a model must check the actual package and the terms of its base model.
Placed in an acceptance table, openness involves at least four layers of rights. Access rights answer whether the organisation can obtain the weights, configuration and documentation. Running rights answer whether it can deploy them on specified hardware and in specified regions. Modification rights cover fine-tuning, quantisation, pruning and replacement of inference code. Transfer rights determine whether modified assets can be supplied to an affiliate, a customer or another service provider. A model may impose few restrictions on the first three layers while constraining the fourth through attribution, scale or base-model terms.
The ability to continue inference must also be separated from the ability to reproduce training. OSI identifies information about the data, data processing and training code, and the parameters as the preferred form for making modifications to a machine-learning system. [2] The Model Openness Framework likewise maps model components to levels of openness. [3] Weights, a tokenizer and an inference implementation are core assets for continued inference, but they are not sufficient by themselves. Model configuration, runtime libraries, conversion tools, drivers, quantisation formats and compatible hardware must also be reconstructible. Reproducing training requires still more: data provenance, cleaning methods, training code, hyperparameters and the computing environment.
Procurement should therefore ask more than whether weights are available. Which components can be obtained? Is the licence compatible with the organisation's way of doing business? Can a modified model be delivered legally? Can its configuration and dependencies be archived? Who will provide security fixes? When the answers are vague, the control created by openness may stop at possession of a downloaded file rather than becoming an executable exit path.
2. Greater control does not automatically produce privacy or continuity
Running a model inside a network controlled by the organisation can materially change the flow of data. Customer-service records, internal code or medical text need not be sent to a third-party API, and the model can be pinned to a revision that has already passed acceptance. Hugging Face Hub's download documentation allows a branch, tag or full commit hash to be supplied as the revision, so the deployment record can state exactly which version of the weights is running. [8] For an organisation that needs an audit trail, a traceable revision is easier to incorporate into change management than an alias that changes over time.
Even so, “only self-hosting provides privacy” is an oversimplification. OpenAI's current data documentation says that, since 1 March 2023, API data is not used by default to train or improve models unless a customer actively chooses to share it. Abuse-monitoring logs are generally retained for up to 30 days, although they may be kept longer where required by law or where necessary to protect the service or third parties. Eligible customers may apply for Modified Abuse Monitoring or Zero Data Retention, subject to approval. [9] Eligibility, endpoints, regions and application-state retention still create exceptions, and public documentation cannot replace the contract and settings that apply to a particular customer.
Data control must cover more than the inference request. Before input reaches the model, it may pass through document parsing, vector retrieval, caches and content filtering. After output leaves the model, it may be written to logs, feedback systems and evaluation platforms. Even when the weights are entirely local, the claim that data stays within the designated boundary fails if any component sends text outside it. Conversely, a hosted API may satisfy business requirements if contracts, network isolation, access controls and retention policies govern the entire path. OpenAI's documentation also says that retention controls require approval, can be configured at organisation or project level, and differ by endpoint in their support for application state and Zero Data Retention. [9] The final evidence should therefore come from the actual data-flow diagram, product settings and contract, not from a “cloud” or “local” label.
Control can be divided further into six dimensions: data, version, behaviour, auditability, continuity and responsibility. An organisation may control the model version without being able to inspect the training data. It may be allowed to fine-tune the model without having the right to redistribute it. It may retain request logs without recording the versions of the tokenizer, quantisation parameters, system prompt and retrieval corpus. Each dimension needs to be assessed separately.
Behavioural control depends particularly heavily on evaluation. Open weights allow a team to fine-tune a model, alter its prompts or add a safety layer, but every modification may affect refusals, factual accuracy, tool use and performance in particular languages. On its first page, NIST's Generative AI Profile states that the AI Risk Management Framework is voluntary rather than a legal requirement. Recommended action GV-1.2-002 includes internal and external evaluation before deployment and on an ongoing basis. [10] An engineering implementation might preserve a baseline before a modification, run regression tests against fixed business examples afterwards, and withhold a release when degradation cannot be explained. That is an implementation recommendation in this article, not a mandatory procedure imposed by NIST on every deployer.
Auditability also requires more than saving inputs and outputs. Explaining a change in behaviour may require the weight revision, tokenizer, inference engine, quantisation parameters, system prompt and retrieval corpus. A full Hugging Face commit hash identifies the file version, but it does not automatically preserve the other runtime conditions. [8] A release that can genuinely be reviewed needs all of that information in the same change record, along with who approved it and when it should be rolled back.
Open weights do not automatically resolve security responsibility either. PyTorch's security guidance warns that models are essentially programmes and that running an untrusted model is equivalent to running untrusted code. It recommends placing untrusted models in an isolated environment and running only software and models from trusted sources. [11] Downloaded packages, serialisation formats, conversion scripts, third-party operators and network-facing service ports all belong in the threat model.
Version pinning has another side. Holding an old commit can prevent changes that have not been evaluated, but it may also miss vulnerability fixes, dependency updates or new business requirements. NIST's voluntary Profile recommends assigning organisational responsibility for periodic review and incident monitoring, and treating routine monitoring as part of continuous improvement. [10] For users, a pinned version should therefore be accompanied by a patching policy, triggers for renewed evaluation, a rollback plan and a maximum period for remaining on that release.
The continuity risk of a hosted model is easier to observe. OpenAI's current deprecation policy generally gives active users advance notice and sets a shutdown date. After that date, the model or endpoint is no longer accessible. Preview models may receive a shorter notice period, while security or compliance issues may require a faster withdrawal; in those circumstances the policy promises notice only where possible. [12] Holding the weights can avoid the closure of one API endpoint, provided that the configuration, dependencies, tools and compatible hardware remain maintainable. It creates another route to continuity, not a guarantee of permanent operation.
3. Cost follows deployment responsibility, not the openness label
One misleading comparison places an API's price per million tokens next to the rental price of a server and attributes the difference to whether the model is open. Openness answers whether an organisation can obtain and modify the assets. The cost structure follows mainly from deployment mode, workload and the allocation of responsibility. The same open weights can be used through a metered managed service or occupy a dedicated instance; proprietary services may likewise offer reserved capacity. GPUs, idle capacity, staff and on-call duties move from the supplier's side into the organisation's total cost of ownership only when the organisation chooses to self-host or assume more platform work.
The three broad modes can first be divided by responsibility. With a hosted API, the supplier manages GPU scheduling, scaling and most model upgrades; the user still manages request permissions, data policy, application evaluation and output risk. A managed open-model service lets the user choose the weights and some runtime parameters while leaving the infrastructure to a cloud endpoint. In a self-hosted deployment, the organisation chooses the hardware, inference engine and upgrade schedule, and also assumes responsibility for capacity, patching, isolation and incident response. These are two separate axes: open versus proprietary describes assets and licensing, while managed versus self-built describes operational responsibility.
The three modes also use different billing units. The cost of a metered API generally follows calls, and an idle application does not keep paying for an entire accelerator. For dedicated endpoints and owned resources, the cost calculation must include running time, peak headroom and idle capacity. Hugging Face Inference Endpoints displays hourly prices by instance specification, calculates actual charges by the minute, and charges while an endpoint is initialising or running. [13] This establishes that a managed open-model service can produce a capacity-based bill. It does not show that this deployment model is widespread, nor that it is necessarily cheaper than per-token pricing.
The shape of requests changes the result further. Long contexts consume more KV cache. Bursts of concurrent requests require greater instantaneous capacity. A strict time-to-first-token target reduces the room for batching. The vLLM paper identifies memory waste and KV-cache management in existing systems as throughput constraints, and explains how its cache manager frees space for batching. [14] A financial model therefore cannot contain only an hourly GPU price. It must also include context length, concurrency, latency, batching and utilisation.
A managed open-model service offers a possible middle course. Hugging Face's Inference Endpoints documentation lists dedicated and automatically scaling infrastructure, security features, monitoring logs and multiple inference engines. [15] A team can switch between open-weight models without building deployment, scaling and logging systems from scratch, while still depending on the platform's network, images, permissions and billing rules. The pricing page states that charges follow the instance specification and running state. [13] Those product facts establish that the option exists and explain how it is billed. They do not establish its adoption rate or a general cost advantage.
Self-hosting begins with establishing that the model works reliably on the target hardware and under the target business workload. The single 80GB GPU and 16GB memory figures in the gpt-oss README describe operation of the MXFP4-quantised versions. They are not guarantees about peak concurrency, context length, quality or response time. [4] NVIDIA's product specifications give the H100 SXM 80GB of memory and a maximum configurable thermal design power of 700W. [16] These numbers can help estimate device counts, power supply and cooling constraints, but they cannot be converted directly into electricity cost, carbon emissions or cost per request. Actual bills also depend on utilisation, data-centre pricing, quantisation method and redundancy requirements.
Inference software can also change the bill substantially. Under the models, hardware and workloads specified in the paper, vLLM reports throughput improvements of two to four times relative to the systems tested. [14] That result shows that serving design can change the throughput delivered by fixed hardware. It does not show that software matters more to cost than model size, quality targets or workload, and it cannot be generalised to every application. Buyers still need load tests based on their own distribution of requests.
Staff cost rises with the responsibilities an organisation assumes, rather than increasing monotonically with openness. Self-hosting or deeply managing a platform requires capacity planning, verification of images and weights, rolling upgrades, metric alerts, backup and recovery, and incident response. With a managed open-model endpoint, the provider still carries part of that work. NIST's voluntary Profile treats pre-deployment and ongoing evaluation, as well as post-deployment monitoring, as recommended actions. This indicates that governance work continues over time, but the Profile neither prices that work nor proves that any deployment mode is cheaper. [10] For an organisation without a platform team, outsourcing, poor utilisation and failure handling may offset saved API charges when it chooses to assume those duties.
Quality cost is also easy to omit. To make a model fit into less memory, a team may use more aggressive quantisation. To increase throughput, it may shorten the context or limit output. The memory guidance for gpt-oss explicitly refers to MXFP4 quantisation. [4] “It fits” is not the same as “it is good enough at the target quality and latency”. Every cost optimisation should be checked against business outcomes: has the error rate risen, are tool calls stable, is long text being truncated, and has the need for human review increased?
A practical comparison needs at least three tables. A request table should record average and peak request volume, input and output tokens, concurrency, latency and failure rate. A resource table should cover GPUs or CPUs, memory, storage, network, redundancy, idle capacity and energy. A responsibility table should assign on-call work, upgrades, compliance review, vulnerability handling, migration and downtime. Each table should be completed for a hosted API, a managed open-model service and a self-hosted deployment, followed by load-testing with real traffic. An organisation can call one mode cheaper only when its own TCO model and workload tests support that conclusion.
4. More choice still has to be engineered into freedom to migrate
Open weights produce one clear change: candidates are no longer constrained entirely by a single supplier's product catalogue. An organisation can pin a model version, choose an inference engine, fine-tune the model on internal data, or give the same weights to different operators. NTIA identifies competition, innovation and research, and broader participation as potential benefits of widely available weights. [1] “Potential” is the important word. More candidates are not necessarily interchangeable, and their existence does not mean the user already has a functioning exit path.
Applications depend on a set of conventions beyond the model itself: how the tokenizer divides text, how a chat template orders roles, where the system prompt is placed, how the JSON schema for tool calls is generated, how large the context window is, and how stop sequences and sampling parameters are interpreted. vLLM provides an OpenAI-compatible HTTP server that can reduce changes to the client interface. [17] The compatibility concerns the shape of the interface. It does not guarantee interchangeability in model capability, parameter semantics, tool behaviour or safety boundaries.
Template requirements also matter when assessing interface-level lock-in. vLLM documents Chat Template separately because the chat protocol must either be obtained from the tokenizer configuration or supplied by the deployer. [17] If the model does not have the correct template, a compatible endpoint cannot automatically generate the correct message format. Migration tests must therefore cover response structure, tool calls, stopping conditions and error handling, rather than merely checking that the port returns an HTTP 200 response.
Lock-in also appears at the infrastructure layer. Once a model is deployed to a cloud provider's dedicated endpoint, the application often depends on that provider's images, private networking, identity permissions, logs, metrics and scaling policies. Moving to another provider requires more than the .safetensors file: deployment scripts, secrets, alerts and performance baselines must move as well. Self-hosting removes one managed layer but may create dependence on a quantisation format, a GPU driver or an internal platform. Lock-in is not a fixed property of either open or proprietary models. It is the number of layers that must be changed during replacement.
An evaluation set is another piece of infrastructure required for genuine choice. Without stable business examples, teams can rely only on leaderboards or subjective trials and may not know which customer workflows a replacement will damage. Deliberately considering more candidates may increase screening costs; that is not a factual claim that open models are released more frequently. An organisation can apply the same thresholds for accuracy, refusal behaviour, latency, cost and security to every candidate. Choice becomes operational only when a second candidate passes those thresholds.
Licences determine whether modifications can travel. Llama 3.1's distribution labelling and terms for certain very large-scale users mean that a new product or legal entity must check the applicable conditions again. [6] For DeepSeek-R1 distilled models, the organisation must trace the relevant Qwen or Llama base-model licence. [7] If a fine-tuning adapter exists only on one platform, or if the training data and evaluation set have not been retained by the organisation, downloadable base weights still do not make the resulting asset easy to migrate.
Open weights, pinned versions and compatible interfaces can reduce some migration barriers, but they do not naturally become leverage in supplier negotiations. That alternative may strengthen the organisation's negotiating position only when another model has passed business evaluation; the weights, configuration, tokenizer, prompt templates, adapters and deployment scripts can be exported; and migration has been rehearsed with a second inference engine or service provider. An untested catalogue of alternatives is only a longer list of names.
5. Choose the deployment model from the constraints
No one mode is superior in every setting. A hosted API suits teams that need to launch quickly, face volatile demand or do not want to maintain a GPU platform, provided that its data policy, region, contract and audit capabilities meet their requirements. A managed open-model service suits teams that want to choose the weights while retaining dedicated instances or automatic scaling; it reduces platform-building work but preserves cloud dependence. Self-hosting suits cases where data must remain within a specified network, the workload has been validated, and the organisation has the necessary platform and security capabilities. It increases control over versions and data location while transferring more responsibility for capacity, patching and incidents to the organisation. These are conditions to test, not a universal ranking.
Eight questions can structure the decision:
• Which data must never leave the organisation's network or jurisdiction, and are logs, caches and backups included?
• Must the business pin a particular model version, and how long an upgrade window can it tolerate?
• Can the weights, configuration, code, tokenizer, data documentation and fine-tuning results be taken away in formats the organisation can read?
• Does the licence permit the intended commercial use, attribution, modification and redistribution?
• Given average traffic, peak concurrency and latency targets, can dedicated capacity achieve an acceptably tested utilisation rate?
• Who is responsible for GPU drivers, the inference framework, vulnerability patches, monitoring, rollback and overnight incidents?
• Does the application have an independent evaluation set and a rollback switch, rather than treating one supplier's output as the only baseline?
• If the supplier or model maintainer ends service or updates, can the organisation move to a second, validated path within the agreed time?
The answers do not have to lead to an entirely self-built or entirely managed estate. An organisation might keep low-sensitivity features with bursty traffic on a hosted API, place batch work that requires a pinned version on a managed open-model endpoint, and run a small number of highly sensitive tasks in an isolated environment. It might also validate demand through an API, test open weights against the same evaluation set, and migrate only when performance, licensing and TCO meet defined thresholds. The purpose of layered deployment is to test data, workload and responsibility separately instead of betting everything on one label.
Governance also needs a responsibility table. Before launch, it should state which assets and fixes the model maintainer releases, which layer of availability the managed provider promises, which patches the enterprise platform team owns, and which outputs and human decisions belong to the business owner. Gaps are more likely when model maintenance, inference hosting and application operation belong to different parties but contracts, upgrade processes or internal ownership do not cover the hand-offs. That is not an inevitable consequence of open models; proprietary services can have the same multi-party supply chain. NIST's voluntary Profile recommends explicit responsibility for periodic review and monitoring, and says organisations can apply controls such as procurement due diligence and service-level agreements to third-party models, data and providers, whether open or proprietary. [10] This recommendation can inform the responsibility table, but it cannot replace contractual interpretation, legal advice, or sectoral compliance obligations.
An exit path also has to be rehearsed. During a pilot, a team can disable the primary endpoint, send a small amount of authorised real traffic to an alternative model, and record recovery time, changes in quality and manual intervention. It can also export configuration from a managed endpoint and redeploy it in a second environment. Such exercises reveal whether the model files, licence, dependencies, secrets, network configuration and evaluation data are genuinely portable. Available weights create a condition for exit; leaving on time still depends on preserving and testing the whole path.
Open models do not merely add entries to a list of “free models”. They change the assets, controls and exit paths that users can arrange. They do not determine who bears deployment responsibility, and they do not guarantee lower cost, greater safety or stronger privacy. A more reliable approach is to express openness as auditable components and licence terms, assign deployment responsibility to named parties, leave cost conclusions to local TCO and load tests, and turn migration into a regular exercise. Provided that configuration, dependencies, tools and compatible hardware remain reconstructible, those preparations improve the feasibility of continued operation or switching. They cannot promise that a model will remain available for ever.
