{"id":6056,"date":"2026-10-26T07:05:46","date_gmt":"2026-10-26T07:05:46","guid":{"rendered":"https:\/\/falcoxai.com\/main\/ai-verification-problem-berkeley-professor-grading\/"},"modified":"2026-10-26T07:05:46","modified_gmt":"2026-10-26T07:05:46","slug":"ai-verification-problem-berkeley-professor-grading","status":"publish","type":"post","link":"https:\/\/falcoxai.com\/main\/ai-verification-problem-berkeley-professor-grading\/","title":{"rendered":"AI Verification Problem: Lessons From a Berkeley Professor"},"content":{"rendered":"

Richard Borcherds, a Fields medalist at UC Berkeley, stopped grading his students’ homework for correctness. Too many were using AI to score top marks, so he now grades on completion and openly calls it “not a very good system.” He has no alternative. If you run quality in a manufacturing operation, that should land uncomfortably close to home, because your controls rest on the same assumption his did: that submitted work proves the person who submitted it knew what they were doing.<\/p>\n

That assumption is breaking down in inspection reports, CAPA write-ups, supplier audits, and validation documentation. This piece covers where the AI verification problem shows up in quality systems, how to redesign checks so they measure outcomes instead of paperwork, and what the shift actually costs versus what it saves.<\/p>\n

A Fields Medalist Stopped Grading Homework Because He Could No Longer Tell Who Knew Anything<\/h2>\n

Note what actually broke. Borcherds did not stop paying readers to check correctness because his students got worse, or better. He stopped because consistent top scores stopped carrying information. The homework still arrived, still looked right, and told him nothing about the person who handed it in.<\/p>\n

That is the AI verification problem in its cleanest form. The artifact survived. The signal inside it died.<\/p>\n

Every quality sign-off in your plant runs on the same logic he just abandoned. A completed inspection report implies someone inspected. A closed CAPA implies someone analysed root cause. A passed competency check implies competence. None of those inferences hold automatically now, and unlike a professor grading on completion, you cannot shrug and call your system “not a very good system” while regulators and customers watch.<\/p>\n

\"Stack<\/figure>\n

What Borcherds Actually Said, and Why ‘Grading on Completion’ Is a Warning Label<\/h2>\n

The interview happened the day after OpenAI announced its agents had apparently solved the Navier-Stokes existence and smoothness problem, one of the hardest open questions in mathematics. That context matters. Borcherds was not reacting to cheating undergraduates. He was reacting to a field where the ceiling moved.<\/p>\n

His current policy is completion credit. Hand it in, get the points. He calls it “not a very good system” and says he has no alternative. That phrase, no good system but no alternative, is where most quality functions already sit without saying it out loud. Your deviation reports get read. Your CAPAs get approved. Nobody checks whether the person who wrote them could have written them unassisted.<\/p>\n

The quote that should go on every operations whiteboard: ‘If AI continues to improve at present rate, then, in a few years, there’ll be nothing human mathematicians can do that AI can’t do better’<\/h3>\n

“If AI continues to improve at present rate, then, in a few years, there’ll be nothing human mathematicians can do that AI can’t do better.”<\/p><\/blockquote>\n

Swap “mathematicians” for “quality engineers” and decide whether you disagree. Most leaders will say the plant floor is different, that physical process knowledge cannot be automated. Partly true. The documentation layer wrapped around that knowledge is not protected at all.<\/p>\n

Borcherds went further. He said he no longer knows whether to advise students to enter the math department, and that he asked ChatGPT the same question and it had no idea either. If the people closest to the technology cannot forecast two years out, your five-year competency development plan is a guess with a template around it.<\/p>\n

Inertia as the default response: Borcherds on paper journals that have been ‘utterly pointless’ for two decades and still publish<\/h3>\n

He described “plenty of inertia” in academia and said he would not be surprised if the math department looked the same in a couple of years. His comparison: physical paper journals, “utterly pointless” for twenty years, still publishing.<\/p>\n

That is the real risk. Not collapse, but procedures that keep running after they stop measuring anything. Your training matrix, your qualification exams, your annual competency reviews. They will keep producing signatures long after the signatures stop meaning competence. Inertia is cheap, which is exactly why it wins by default.<\/p>\n

The Same Breakdown Is Already Happening in Quality Sign-Offs and Supplier Documentation<\/h2>\n

A deviation write-up used to be two things at once: a record of what happened, and proof that somebody sat with the problem long enough to explain it. The second function is gone. An 8D report generated in four minutes reads cleaner than one a line engineer spent two days on, and it lands in the reviewer’s queue with the same weight.<\/p>\n

The failure mode is specific. The document clears review, the thinking behind it was never done, and nobody learns otherwise until the same defect comes back on the same line with the same containment action already on file. By then the corrective action register says the issue was closed.<\/p>\n

Why polished documentation now correlates with less scrutiny, not more<\/h3>\n

Reviewers have always used writing quality as a proxy for rigour. Tight prose, structured root-cause logic, correct terminology, these signalled someone competent. Generated text clears that bar by default, so the signal inverts: the better a supplier audit response reads, the faster it gets approved and the less likely anyone probes the evidence underneath.<\/p>\n

Watch your own approval times. If CAPA narratives and SOP revisions are getting shorter review cycles while recurrence rates hold steady or climb, you are approving faster because the documents look better, not because the work improved.<\/p>\n

The competency gap nobody is measuring: reviewers who cannot tell authored work from generated work<\/h3>\n

Borcherds stopped paying readers to grade for correctness because he judged there was “no point” to it. He at least knew his verification had stopped working. Most QA organisations have not run that test on themselves, and their reviewers are approving documents with no ability to distinguish authored reasoning from fluent output.<\/p>\n

This is measurable, and you should measure it. Pull ten closed CAPAs from the last quarter, bring the named authors into a room, and ask them to explain the failure mechanism without the document in front of them. The gap between what the paperwork claims and what the author can reconstruct is your real exposure.<\/p>\n

\"Quality<\/figure>\n

Redesigning Verification: Check Outcomes and Reasoning, Not Submitted Artifacts<\/h2>\n

Borcherds is stuck because universities have only one place to look: the artifact. A homework set is the entire evidence base. You are not stuck, because your work produces physical results. Scrap rates, recurrence, first-pass yield, and customer complaints are all verification signals that no model can write its way around.<\/p>\n

So stop asking whether a correct-looking document was produced. Ask whether the defect rate moved, whether the failure came back, and whether the person who signed the report can defend the reasoning out loud.<\/p>\n

A four-step verification redesign you can run on one process this quarter<\/h3>\n

Pick one process. Root-cause investigations on a single line work well, because recurrence data already exists and the review load is painful enough that people will cooperate.<\/p>\n

    \n
  1. Add a five-minute verbal defence<\/strong>: before any root-cause report closes, the author walks the reviewer through why the chosen cause explains the data and why the alternatives were ruled out. No slides. Questions allowed.<\/li>\n
  2. Instrument effectiveness with recurrence, not closure<\/strong>: track whether the same failure mode reappears on the same line within 90 days. Closure counts tell you nothing.<\/li>\n
  3. Declare AI use in the template<\/strong>: a mandatory field stating what was drafted with AI and what was verified by hand. Policing is a losing game. Disclosure is not.<\/li>\n
  4. Reallocate freed review hours<\/strong>: send them to the investigations with the highest recurrence risk, not evenly across the queue.<\/li>\n<\/ol>\n

    What the ROI looks like: fewer review hours per report, measured against recurrence and scrap, not document throughput<\/h3>\n

    Two numbers move in opposite directions, and that is the point. Review hours per document drop, because reviewers stop line-editing prose and start interrogating logic. Recurrence-driven rework, scrap, and customer escalations become the metric leadership actually watches.<\/p>\n

    Document throughput should be removed from the dashboard entirely. It was always a proxy, and AI has made it a worthless one. A plant closing fewer investigations with lower repeat failure rates is outperforming one closing twice as many.<\/p>\n

    \n

    Ready to find AI opportunities in your business?<\/strong>
    \nBook a Free AI Opportunity Audit<\/a>. It is a 30-minute call where we map the highest-value automations in your operation.<\/p>\n<\/div>\n

    Building Controls That Survive the Next Capability Jump<\/h2>\n

    Borcherds expects his department to look roughly the same for a couple of years. He described the academic system as full of “inertia,” and compared paper journals that keep publishing despite being “utterly pointless” for two decades. He is probably right. Universities can absorb a broken grading system for a long time because nothing explodes when the signal dies.<\/p>\n

    You do not get that. A quality control that stops carrying information shows up as a return, a recall, a customer escalation, or a finding in your next audit. The clock between “our verification stopped working” and “a customer found out” is measured in weeks, not years. Inertia is a luxury of institutions whose output nobody tests.<\/p>\n

    Policy statements are not controls: what to put in place instead<\/h3>\n

    Berkeley’s answer so far is a syllabus statement. Faculty are asked to describe course expectations around AI tool use, and that is the extent of the instrument. Most manufacturers are heading the same way: an AI use policy in the quality manual, a disclosure checkbox on the document template, a line in the training deck. None of that is a control. It is a statement of intent that produces no evidence and fails no one.<\/p>\n

    A control has to generate a signal you can read without trusting the author. Three that work:<\/p>\n