Between August 27 and September 9, 2026, 155 responses from Cloudflare’s GLM-5.3 Workers AI model contained data from other customers’ prompts. The cause was not a rogue model or a clever attacker. Cloudflare deployed the model with a 1,048,576-token context window while the KV cache pool on its prefill GPUs only had room for 681,280 tokens. They knew about the mismatch before launch and assumed the engine would throw an error. It didn’t. It handed back stale cache pages holding someone else’s session.

If your team is sending production data into hosted AI endpoints, this is the failure mode that should worry you, and it won’t show up in a vendor’s AI ethics statement. Below, the questions to put to your AI providers and the prompt-data rules worth writing down before your next deployment.

155 Responses Carried Other Customers’ Data, And Nobody Attacked Anything

Cloudflare’s own write-up is blunt about who found this: a security researcher, Gadi Evron from Knostic, filed a HackerOne report on September 8. Cloudflare reproduced it the next morning and pulled the model within 40 minutes of declaring an incident. After excluding their own testing and the researcher’s, the affected requests looked “consistent in timing and volume with ordinary long-context use, not with someone repeatedly exploiting the bug.”

No jailbreak. No prompt injection. No adversarial actor probing for weaknesses. Just regular customers running regular long prompts and getting someone else’s session back.

That is the part worth sitting with. Your supplier contract terms, your deviation reports, your process parameters do not need an attacker to end up in a competitor’s output window. They just need a mismatched configuration and an error path that failed quietly.

What Actually Broke: A Context Window That Didn’t Match the Cache

Two things had to go wrong at the same time, and neither of them involved the model producing bad output. The first was a configuration mismatch. The second was an inference engine that handled that mismatch by lying about its own state.

Why the long-context tests passed and hid the problem

Cloudflare has been upfront that they knew the context window and the cache pool did not line up before the model went live. Their assumption was reasonable on paper: send the engine a prompt it cannot fit, and it returns an error. That is how most systems behave when they run out of room.

SGLang did not do that. It marked the over-length request as rejected internally, then quietly substituted a one-token placeholder and carried on. Because a response came back, the long-context tests looked like passes. The test suite was checking that a request succeeded, not that the engine had actually processed the prompt it was given. That gap is the part worth copying into your own validation checklist.

Disaggregated prefill/decode: the shared-state assumption that failed

Modern inference splits the work across machines. Prefill GPUs build the key-value cache for a prompt, then hand it to decode nodes that generate the tokens. The handoff depends entirely on a message saying the transfer is done and the cache is current.

That message arrived. The cache behind it did not. Prefill generated and shipped the cache for the one-token placeholder, the decode node read “transfer complete” and assumed the previous request’s state had been overwritten, and then generated a response while attending over stale pages that still held earlier sessions. No boundary check, no verification that the cache size matched the prompt size. The KV cache leak was a trust assumption between two services, not a security flaw in the model.

Every hosted AI endpoint you use has a version of this handoff running inside it.

The Response Timeline Is the Part Worth Copying

Containment here was measured in hours. Cloudflare shipped a corrected configuration and a patched inference engine and restored the model on September 9, the same day they reproduced the report. Plenty of vendors would have quietly rolled back a config, said nothing, and let the affected customers find out never.

Two details matter more than the speed. Cloudflare published its report as a coordinated disclosure alongside SGLang, the open-source inference engine behind the fault, and committed to working with upstream maintainers on this and similar bugs. They also stated plainly that no other model hosted on Workers AI was affected, which means they had enough instrumentation to check. That is the standard. Judge your vendors against it.

Five questions to put to any hosted-model vendor before signing

None of these require a security background to ask. They require a vendor willing to answer in writing.

• How is tenant isolation enforced at the inference layer?: Not at the API gateway. Inside the engine, where cache is pooled across requests.
• What is your time-to-containment for a confirmed data exposure?: Ask for a number and an example. Forty minutes from incident declaration to model offline is an answerable bar.
• Which open-source components sit in the serving path, and who patches them?: If they cannot name the inference engine, they cannot tell you what it does when a request exceeds its limits.
• Will you notify us of exposures affecting our account, and in what window?: Get this in the contract, not the trust page.
• Can you reconstruct which of our requests were affected?: Vendors who cannot scope an incident will tell you everything is fine because they genuinely do not know.

Vendors who answer these quickly are the ones already running incident drills. Vendors who route you to an AI ethics statement or a SOC 2 report are answering a different question than the one you asked. Make the answers part of the procurement file and review them when the vendor changes its serving stack.

What This Changes for Manufacturing and Quality Teams Using Hosted Models

Look at what your teams actually send to hosted models. Deviation reports. Draft CAPA write-ups. Supplier drawings and spec sheets. Audit evidence packs before the auditor sees them. Almost none of that is public, and most of it arrives with customer names, batch numbers, or process parameters attached.

You do not fix this with an AI ethics statement. You fix it with a data classification rule, a vendor questionnaire, and a short playbook. That is a few days of policy work, and it is usually the difference between a security team blocking your AI roadmap and signing off on it.

Tiering your prompt data: what can go to a shared endpoint

Three tiers is enough. Tier 1 is general text with no identifiers, which can go to any shared-tenant endpoint. Tier 2 is internal operational content, which goes to shared endpoints only with identifiers stripped. Tier 3 covers regulated records, customer-confidential specs, and anything tied to a validated process. Tier 3 goes to dedicated or on-prem inference, or it does not go at all.

One specific rule worth adding: keep Tier 3 content out of very long prompts. The affected responses were all to prompts longer than 681,280 tokens, because long context is where cache management gets complicated. Pasting a 400-page quality manual into a shared endpoint is exactly the pattern that breaks first.

Building a 24-hour exposure assessment playbook

When a vendor publishes a disclosure, you need to answer one question fast: did we send anything to that model, in that window, that we care about. You cannot answer it without prompt-level logging. Log the model name, timestamp, prompt length, and a data-tier label on every call. Retain for 90 days minimum.

Write the rest down on one page. Who checks the logs, who decides whether it is reportable, who calls the affected customer or notifies the notified body. Rehearse it once. An untested playbook is a document, not a control.

The Shared-Infrastructure Trade-Off Isn’t Going Away, Price It Properly

Cheap inference is built on shared state. Disaggregated prefill and decode, aggressive cache reuse, batching across tenants: these are the optimisations that make a million-token context window affordable in the first place. Every one of them puts your request in the same memory neighbourhood as someone else’s.

Expect more faults of this shape as context windows keep growing. The engineering surface gets wider, the test matrix gets harder, and the assumption Cloudflare made (that an over-length prompt returns a client error) is the kind of assumption every provider makes somewhere in their stack. Avoiding hosted AI is not the answer. Pricing the trade-off is.

Three things belong in that calculation. Match data sensitivity to deployment model rather than picking one tier for everything. Treat vendor transparency as a selection criterion, because a provider that publishes a coordinated disclosure with its upstream maintainers is telling you something a provider with silent rollbacks never will. And keep a tested fallback, because a model pulled offline for a day should not halt a release decision or a batch disposition.

When dedicated or on-prem inference earns its cost

Dedicated endpoints and on-prem inference cost more per token, sometimes several times more. They earn it when the data carries contractual or regulatory weight: customer-identifiable deviation data, unreleased product specifications, supplier agreements under NDA, anything that would trigger a notification obligation if it surfaced in a stranger’s session.

They do not earn it for the bulk of day-to-day work. Summarising internal meeting notes, drafting procedure text, classifying maintenance tickets with identifiers stripped: run that on shared infrastructure and accept the residual risk, because the alternative is paying premium rates to protect material that was never sensitive.

The split only works if someone decides which bucket each workflow falls into before the integration is built. That is how we structure deployment decisions: data classification first, architecture second. Retrofitting a classification scheme onto six live integrations costs more than doing it on day one, every time.

Source: blog.cloudflare.com