Somewhere on your shop floor, an open-weight model is already running. Maybe someone on the quality team pulled it down to draft deviation reports. Maybe IT spun it up for document search. Nobody signed off, because it was free and it worked. Then Cohere, a company that builds enterprise models and is buying security firm AISLE to push cyber-resilient sovereign AI, flagged concerning behaviour in one of the most widely used open-weight models. That flag matters less as a security headline than as a question: who approved the model you are running?

Open-weight AI model risk is a governance gap, not a hacking problem. Below, we break down what the warning actually signals, how to inventory the models already in your plants, and what a practical vetting process looks like for a team without a dedicated AI function.

Nobody on Your Team Can Name the Model Behind Your Quality Assistant

Ask your plant manager which model powers the assistant drafting CAPA summaries. Then ask your MES vendor what sits behind their new “AI insights” tab. In most manufacturing organisations, both answers come back vague, and nobody treats that as a problem. Open-weight models spread the way spreadsheets did in the nineties: fast, informally, and with no owner.

So when a vendor like Cohere, which sells enterprise platforms such as North and Compass and offers on-prem and isolated VPC deployments, publicly flags concerning behaviour in a popular open-weight model, most operations teams have no way to check whether they are exposed. No inventory. No owner. No mechanism.

Open weights are not the problem. Plenty of serious manufacturers run them deliberately, in isolation, for good reasons. Unmanaged open weights are the problem.

What Cohere Reported and Why Open Weights Change the Risk Profile

The substance of Cohere’s warning is narrow: a company with real enterprise deployment experience observed concerning behaviour in a widely adopted open-weight model and published it, alongside pointers to its own research arm, Cohere Labs, and its Model Vault and Private Deployments products. Read the positioning honestly. A vendor selling isolated inference has an interest in you worrying about models you downloaded for free.

That does not make the point wrong. The structural difference between a closed API and a set of weights on your hardware is real, and most manufacturers have never thought it through.

Who owns model behavior once the weights sit on your servers

When you call a hosted API, you are renting a moving target. The provider monitors usage, patches behaviour, retires bad checkpoints, and absorbs the liability of running the thing. You get upgrades you did not ask for, which is annoying until the day it saves you.

Download the weights and that relationship ends. The file does not update. Whatever behaviour was baked in during training stays in, permanently, on your infrastructure, and you are the one who has to find it. There is no vendor issuing a fix and no changelog telling you something shifted. You inherited a frozen artefact and the full responsibility for what it does.

Why ‘we run it on-prem’ is a security claim, not a safety claim

On-prem deployment answers one question well: does our process data leave the building? For a plant handling customer specifications or regulated batch records, that matters enormously. It is also the only question on-prem answers.

It says nothing about whether the model fabricates a root cause in a deviation report, misreads a tolerance, or behaves unpredictably on inputs nobody tested. Air-gapping a flawed model just means the flaws stay private. Data residency and model behaviour are separate controls, and conflating them is how an IT sign-off ends up standing in for a quality assessment that never happened.

Where Open-Weight Models Still Earn Their Place in Manufacturing

None of this is an argument for ripping them out. Open weights are still the only workable answer for an air-gapped plant, for document parsing over controlled IP you are contractually barred from sending outside, and for any workload where data residency rules kill an external API before the conversation starts. Cohere itself sells manufacturing as an industry vertical and offers on-prem deployment, which tells you the demand for local inference is real, not a hobbyist preference.

Draw the boundary by consequence, not by enthusiasm. If a human or a deterministic system checks every output before it goes anywhere, model quirks are a nuisance you can absorb. If the output writes to your QMS, triggers a deviation, or lands in an audit trail a regulator will read, a model nobody formally approved has no business in that path.

Low-stakes, high-volume work where open weights pay for themselves

The economics are clearest on classification at volume. Routing thousands of maintenance tickets by equipment type, tagging supplier emails, clustering free-text operator notes, extracting fields from scanned COAs before a human validates them. These tasks run constantly, cost real money per call on a commercial API, and fail safely because a wrong label gets corrected downstream rather than propagating into a record.

Same logic applies to retrieval. Indexing twenty years of work instructions, SOPs, and equipment manuals so an engineer can find the right paragraph in ten seconds is a search problem with a human reading the answer. Nobody acts on the output without opening the source document.

Translation and summarisation of internal material sit in the same tier, as does first-draft generation that a quality engineer will rewrite anyway. The common thread is reversibility. Pick workloads where a bad output costs someone four minutes, not a 483 observation. Then document which model serves each one, so the decision survives the person who made it.

A Vetting Protocol to Run Before a Model Touches Quality Data

The protocol is five steps, and none of them require a data science team. Inventory every model in use. Assign a named owner to each. Test behaviour against your own cases. Log inputs and outputs wherever a model influences a quality decision. Define a replacement path before you need one.

The model inventory most manufacturers have never built

Start with what you can see: anything running locally, anything someone installed, anything a developer wired into a script. Then go after the harder half. Every vendor tool with an “AI” feature is running a model you did not choose, and your contract probably says nothing about which one.

Ask each vendor three questions in writing: which model, which version, and will you notify us when it changes. Cohere’s own product line shows how granular this gets, with Command for generation, Embed for semantic representation, Rerank for retrieval, and Parse for document parsing. A single “AI feature” in your MES could be four models stacked. One owner per model, named in the inventory, with authority to pull it.

Testing against your deviations, not public benchmarks

Public benchmarks tell you nothing about whether a model handles your deviation language. Build a test set from fifty real cases out of your own history: ambiguous root causes, contradictory operator notes, the batch record with the handwritten correction. Run every candidate model against the same set and score the outputs yourself.

Rerun that suite on every version change. This is the part teams skip, and it is the part that catches silent regressions when a vendor swaps a model underneath you. Pair it with full input and output logging so an auditor asking “how did the system reach that conclusion” gets an answer instead of a shrug.

Cost of the protocol: a few weeks of someone’s time, plus logging infrastructure you probably already have. Cost of one contaminated batch record traced to an unvetted model, or one audit observation on data integrity: investigation, remediation, and a regulator who now looks harder at everything else.

What This Changes for Your 2026 AI Roadmap

Expect more disclosures like this one. Open-weight adoption has run ahead of oversight in almost every manufacturing organisation, and the vendors with enterprise deployment experience are the ones positioned to notice when something behaves oddly. Cohere’s acquisition of AISLE, framed around “cyber-resilient sovereign AI”, signals where the commercial attention is going. More eyes on model behaviour means more findings, and more findings published.

The second shift is external pressure. Your notified body, your automotive customer’s supplier audit, your ISO surveillance visit: all of them will eventually ask which model generated a document in your quality system, and who validated it. Today that question lands as a curiosity. Within a year or two it lands as a finding. “A tool the engineering team installed” is not an answer that survives an audit trail review.

None of that is a reason to slow down. The teams that stay fast in 2026 are the ones who treated model choice as a reversible, documented decision from day one. If you know what you are running, who owns it, and what you would swap to, a disclosure like Cohere’s becomes a Tuesday afternoon task instead of a shutdown. If you do not know, every warning triggers the same scramble, and the scramble costs you more than the governance would have.

Build the ability to change your mind cheaply. That means prompts and retrieval logic kept separate from the model itself, outputs logged in a form you can re-run, and at least one tested alternative for anything touching a regulated process. Lock-in is the real risk, not any single set of weights.

One action worth taking this quarter: build the model inventory and put a named person against each entry. Not a department, a person. It takes a few weeks of asking uncomfortable questions across IT, quality, and your vendor list, and it costs nothing but attention. Everything else in your AI governance programme depends on knowing what is already running.

Source: cohere.com