Ten minutes with ChatGPT was enough. In a peer-reviewed study presented at the Conference on Language Modeling, researchers including UC Berkeley’s Brian Christian ran randomized trials with 1,222 participants. One group solved fraction problems alone. The other had AI on the side. The AI group scored higher at first. Then researchers took the tool away after 12 problems, and accuracy collapsed almost immediately. People gave up. The unassisted group kept going and finished stronger. A third experiment with SAT reading comprehension produced the same pattern.

If your quality or operations team is rolling out AI assistants, this should change how you design the rollout, not whether you do it. The study is a specification. Below, what AI skill erosion actually looks like on a factory floor, and how to structure workflows so your team’s capability compounds instead of quietly draining away.

Your Team Is Faster With AI. What Happens the Day the Tool Is Gone?

Run the scenario. Your deviation assistant is down, the vendor pushed an API change overnight, or a complaint lands that sits outside anything the model has seen. Who on your team still works the problem to the end? Most leaders assume the answer is “everyone, just slower.” The Berkeley research says otherwise. People with AI support did not degrade gracefully when it was removed. They quit.

That matters more in quality and manufacturing than almost anywhere else. Root cause analysis, CAPA investigations, and audit defense all run on stamina: the willingness to stay with an ambiguous problem past the point where the obvious answer fails.

Throughput gains you can measure this quarter. AI skill erosion shows up the one week you cannot afford it.

What the Berkeley Persistence Study Actually Measured

Brian Christian, a research fellow at the UC Berkeley Center for Human-Compatible AI and author of The Alignment Problem, co-authored the work. This was not an opinion piece about screen time. It was a set of randomized controlled trials, peer-reviewed, with participants recruited online and assigned to conditions at random.

The design was deliberately boring, which is what makes it useful. Give people a task. Give half of them help. Take the help away. Watch what happens next.

The three experiments: 354 participants on fractions, 667 on a replication, 201 on SAT reading comprehension

In the first trial, 354 people worked through 15 basic fraction problems. One group had nothing but the problems. The other had ChatGPT open alongside, free to ask for hints or simply request the answer outright. After 12 problems, the tool was pulled.

The second trial repeated the setup with 667 participants, nearly double the sample. The third swapped arithmetic for an SAT reading comprehension prompt and ran 201 people through it. Three different task types, three different cohorts, one consistent pattern. Replication at this scale is rare in this field and it is the reason the finding carries weight.

The finding that matters most: accuracy dropped, but so did willingness to keep trying at all

The accuracy drop is the headline most coverage picked up. It is the less interesting half. What the data actually showed is that participants who lost their AI did not grind slowly toward wrong answers. They abandoned questions. The rate of giving up spiked.

The unassisted group, meanwhile, stayed with the task and finished more successfully than the people who had started out ahead. Christian’s own summary of the result is blunt:

You almost couldn’t ask for a clearer story.

That distinction between skill and stamina is the whole point. AI skill erosion shows up first as a collapse in effort, not knowledge. Your team may still know how to run the investigation. The question is whether they will still sit with it when it gets hard.

Why Persistence Is the Exact Skill Quality and Operations Work Depends On

Quality work is iterative by nature. The first hypothesis in a root cause analysis is almost always wrong. So is the second. The value shows up on the fourth or fifth pass, when someone goes back to the batch records, re-reads the operator interview, and notices the shift change that nobody flagged.

That is the muscle the Berkeley trials measured. Not knowledge, not speed, but willingness to stay in an unresolved problem long enough to get somewhere useful. Christian keeps a notebook on his desk and writes in it every day, specifically to tamp down the suspicion that too much time working with AI could cost him his own scholarly edge. He co-authored the study and he still hedges against it personally. That should tell you something about how seriously to take this.

Here is the uncomfortable part. AI is being deployed hardest on exactly the tasks that used to build persistence: summarising deviation reports, drafting CAPA narratives, first-pass trend analysis, pulling together audit prep packs. The hard escalations stay human. We have automated the reps and kept the exam.

The tasks where giving up early is a compliance risk, not just a productivity dip

A sales team that gives up early loses a deal. A quality team that gives up early signs off a CAPA with a symptom labelled as a root cause. That closure is a document. It goes into the file, it gets pulled in the next audit, and it sits there as evidence that your investigation process does not actually investigate.

Deviation triage carries the same exposure. The difference between “operator error” and a genuine systemic finding is usually three more questions nobody felt like asking. Effectiveness checks, recurring complaint patterns, supplier escalations: all of them reward the person who refuses to accept the plausible answer. When AI dependency at work quietly lowers the threshold for what counts as “done,” the cost lands in your next regulatory inspection, not your weekly throughput report.

Designing AI Workflows That Keep Capability Intact

The fix is not restricting access. It is sequencing. The trials failed the AI group because the model spoke first and the human never had to form a position. Reverse that order and you keep the speed without hollowing out the people.

Four workflow patterns that put the human attempt before the AI answer

These are process rules, not software features. You can enforce all four in your existing QMS with a template change and a bit of discipline from the people who approve investigations.

• Attempt-then-assist: The investigator writes a first hypothesis and a proposed line of inquiry before the assistant is opened. Short, five minutes, logged in the record.
• Tool-free intervals on high-judgment work: Complaint triage, risk assessments, and anything heading to a regulator gets a defined no-AI first pass.
• Justify the accept or reject: Every AI-generated root cause or CAPA draft requires one line explaining why the human kept it or overrode it. No blank approvals.
• Rotate escalations: If the same two people handle every non-standard case, capability concentrates there and erodes everywhere else. Rotate deliberately.

Brian Christian’s own habit is instructive here. He keeps writing by hand daily precisely because he works with AI constantly. Build the equivalent into the job, not into people’s good intentions.

The metrics to track alongside time saved, so erosion shows up before it costs you an audit

Cycle time will improve immediately and it will tell you almost nothing about capability. Track it, then put three harder measures next to it: investigation depth (distinct hypotheses tested before closure), first-time-right rate on CAPA effectiveness checks, and recurrence of the same failure mode within twelve months.

Add one stress test. Twice a year, run a live deviation with the assistant switched off and measure how far the team gets unaided. The Berkeley group found people “gave up entirely” when the tool was withdrawn. Better to learn that in a drill than in front of an auditor.

The Capability Line Item Missing From Your AI Business Case

Almost every AI business case approved in the last two years counts one thing: hours removed. Deviation write-up time down 40 percent. Supplier audit prep cut in half. Those numbers are real and they belong in the model. They are also incomplete.

If the same deployment that saves two hundred hours a quarter degrades the judgment of the five people who handle your worst nonconformances, the net is negative. It will not show up on the dashboard, because no dashboard tracks it. You will find out during a regulatory inspection or a customer escalation, which is the most expensive possible time to learn it.

A defensible business case going into 2027 carries two numbers, not one. Efficiency gain and capability retention, measured separately, reviewed in the same meeting. When the second number moves the wrong way, the first one stops counting as a win.

What to add to your next AI deployment review so capability loss is a tracked risk, not a surprise

Treat it the way you treat any other process risk. Name it, assign an owner, set a measurement method, and put it on the review agenda with a defined frequency. Brian Christian called the finding “a striking finding, but in a way it supplies ammunition for a story that a lot of us kind of feel in our gut.” Gut feel does not survive a governance review. A tracked metric does.

• An unassisted baseline: Before go-live, capture how your team performs on a representative problem without the tool. You cannot detect erosion without a starting point.
• A named capability owner: One person accountable for whether investigators can still work a problem end to end.
• A periodic tool-off exercise: Quarterly, on a real closed case, with results logged.
• A stop condition: Define in advance what level of decline triggers a workflow redesign rather than a wider rollout.

Four line items. None of them slow down adoption. All of them make capability visible before it costs you something.

Source: vcresearch.berkeley.edu