Data engineer Simon Späti put it bluntly in a September 2026 post: the problem is not the AI code, it’s that nobody knows anything anymore, and everyone just asks Claude. He quoted an engineer at a large company describing teams of 12-hour days spent pressing enter on specs, tickets, tests and reports nobody reads. Code still shipped. Understanding of the AI system architecture behind it quietly evaporated.

That same pattern is landing in operations and quality functions right now. Your models work, your dashboards refresh, and then the person who knew why a threshold was set at 0.85 leaves. This article covers where design intent actually disappears, the documentation and review habits that keep it in the building, and what it costs you when it’s gone.

Nobody Knows Anything Anymore: The Quiet Failure Mode of AI Adoption

Späti’s sharpest line is the one people skip: you end up with no plan whatsoever. The output passes. The plan is gone. The engineer he quoted described it from the inside.

There is no sense of victory. Nobody is resolving bugs. In reality, nobody is thinking anymore. Everything is done by LLMs.

Swap “code” for “documentation” and you have most AI adoption in manufacturing today. SOPs drafted by a model, inspection logic tuned by a model, deviation reports and validation packages assembled in minutes. All of it defensible on paper. None of it carrying the reasoning behind it.

Then an auditor asks why the rework rule triggers at that specific point, and the room goes quiet. That is the failure mode. Not a wrong answer, but no one who can defend the right one.

What Actually Disappears: Architecture Knowledge and Design Intent

Two different things go missing, and treating them as one problem is why most documentation efforts fail. The first is structure: which systems exist, what feeds what, which model reads which sensor, where the handoff to the MES sits. The second is intent: why that tolerance band, why this data source instead of the historian, why a manual sign-off survived three rounds of automation.

Structure is recoverable. Intent is not. And intent is the thing auditors, customers and your own engineers actually need when something drifts.

Why AI can regenerate structure but not reconstruct reasoning

Point a model at your pipelines, scripts and config files and it will produce a serviceable map. Späti’s observation about code applies exactly here: if your code base was below average AI can easily improve it up to average. The ceiling was never the issue. Average structure documentation, generated on demand, is fine.

Reasoning is different because it was never written anywhere. The decision lived in a meeting, a supplier complaint, a failed batch in 2019. A model has nothing to read, so it produces something plausible instead. Plausible reasoning in a quality system is worse than a blank field, because nobody challenges it.

The compounding cost: every undocumented decision becomes a future audit finding

Undocumented intent does not stay neutral. It turns into a change you cannot justify, a deviation you cannot close, a validation package that survives the audit only because the auditor did not push. Each one raises the cost of the next change, because engineers hedge rather than touch logic they do not understand.

The compounding is quiet. Year one, two or three decisions lack a trail. Year three, nobody modifies the inspection model at all, so it stays tuned for a product mix you stopped running. Design intent documentation is cheap at the moment of the decision and close to impossible to recover eighteen months later, once the people involved have moved on.

Why Quality and Manufacturing Teams Are More Exposed Than Software Teams

Hoyt Emerson makes the case that data people are insulated from this:

I think Data people are different. We’ve had to know everything about the product/business from day 1. AI just removes friction for us now.

Späti’s response is the one worth sitting with. That knowledge is only seemingly obsolete. Anyone starting in a new field today prompts their way through it and never builds the underlying model of how the process works. The friction Emerson is glad to lose was the thing doing the teaching.

The difference between a rollback and a recall

Software teams get a safety net that quality teams do not. A bad feature ships, someone reverts the commit, and by Monday it never happened. The cost of not understanding your own system is measured in hours.

A batch release decision does not revert. Neither does a CAPA closed on flawed reasoning, a validated inspection rule that passed the wrong parts for six weeks, or product already sitting at a customer site. You cannot roll back a shipment. That asymmetry is why the same AI adoption pattern that produces mild chaos in engineering produces regulatory exposure in manufacturing.

What an auditor asks that a prompt history cannot answer

Auditors do not ask what your system does. They ask why it does it that way, who decided, on what evidence, and what alternatives were rejected. ISO 9001 and FDA reviewers are testing whether a competent person made a defensible decision. “The model suggested it” fails that test immediately.

A prompt log proves a conversation happened. It does not prove judgment was applied. If your justification for a control limit is a chat transcript, you have documentation of a process, not documentation of a decision, and an auditor will read the difference in about thirty seconds. The rule is simple: every parameter in a regulated system needs a named human who can defend it without opening a tool.

Keeping Intent Explicit While Still Moving Fast With AI

None of this argues for slowing down. It argues for three operating rules that cost almost nothing and hold the line on understanding while your team uses AI as hard as it wants.

The one-paragraph decision record that survives staff turnover

Every AI-assisted decision that touches quality, safety or a customer outcome gets one paragraph of written rationale, stored with the artefact itself, not in a separate wiki nobody opens. Name the alternatives you rejected and why you rejected them. That second half is the part teams skip, and it is the only part that matters six months later.

Keep it to a paragraph on purpose. Long templates get filled with model-generated filler and stop being read. A specification change, a retrained model, a revised acceptance limit, each carries a short note that a new hire can read in thirty seconds and understand the tradeoff that was live at the time.

Named system owners and AI-free architecture reviews

Assign one person per system who must be able to draw it on a whiteboard, from sensor to sign-off, without opening a tool. If they cannot, you do not have an owner, you have a contact. Run that test quarterly. It takes fifteen minutes and tells you more about your exposure than any documentation audit.

Then run architecture walkthroughs where AI is banned from the room. No prompting, no retrieval, just people explaining their own systems to each other. Sean Behan’s point applies directly here: knowing what you want has always been the hardest part. A capable non-coder can now build anything, and will still build on a bad foundation if the mental model underneath is wrong.

The economics favour this. Teams that write down intent spend less on rework, onboarding and audit preparation than teams that ship faster and rediscover their own logic every quarter. Rediscovery is unbudgeted, unplanned, and always arrives during an audit or a customer complaint.

The 2026 Advantage Goes to Teams That Still Understand Their Own Systems

Output speed is levelling off as a differentiator. Everyone with a budget can generate documents, draft procedures and produce reports at roughly the same rate now. The managers in that engineering thread already found the ceiling: pushing code stopped being the bottleneck, and the work got slower anyway, because nobody could explain what had been built.

What is left to compete on is explanation. Can you say why the system behaves the way it does, defend that reasoning to a customer auditor, and change it on purpose rather than by re-prompting until the output looks right? Teams that can do this move faster on the second and third iteration, which is where the actual cost sits.

Plan for the next twelve months around one combination: AI-assisted throughput plus enforced intent capture. Throughput alone gives you a documentation set that collapses under the first serious question. Intent capture alone makes you slow. Together they scale past the first team, survive turnover, and keep your senior people making judgement calls instead of spending thirteen hours a day pressing enter.

The questions to ask your team this quarter

Pick your three most AI-dependent processes and put the same questions to whoever owns them. Not in a review deck. In a room, out loud, with no tool open.

• Why is this parameter set here?: If the answer is “that’s what came out”, you have a gap, not a system.
• What did we reject?: A design with no rejected alternatives was never designed.
• Who can change this safely?: Name a person. If the honest answer is “whoever prompts it next”, fix that first.
• What breaks if the input source changes?: Structure questions are answerable. If yours are not, start there.

Write down what people could not answer. That list is your actual risk register for AI adoption in manufacturing, and it is more useful than any model performance report you will run this year.

Source: ssp.sh