Cloudflare's GLM 5.3 leaked other users' data into 155 replies through a cache bug

For almost two weeks this summer, a very long prompt sent to one of Cloudflare's hosted AI models could come back with somebody else's data in the answer. Cloudflare disclosed the problem on Monday, October 26, in a detailed incident report on its blog. Between August 27 and September 9, 2026, the GLM 5.3 model on Workers AI returned 155 responses that contained data from other customers' requests made to the same model shortly before. Every one of those responses was an answer to a prompt longer than 681,280 tokens.

This is not a story about a clever hacker or a model that went rogue. It is a story about plumbing. Nothing was stolen through a jailbreak, and the model itself did nothing wrong. Two ordinary mistakes stacked on top of each other inside the serving stack, and the result was the kind of failure every company that sends sensitive text to a cloud model quietly worries about: tenant isolation breaking, without any error message, in a response that looked completely normal.

What happened, in plain terms

GLM 5.3 is a text model from the Chinese lab Z.ai that is built for agentic coding work. Cloudflare offers it on Workers AI under the name `@cf/zai-org/glm-5.3`, next to many other open models. It launched there on August 27 with a context window of 1,048,576 tokens, which is roughly a million tokens of input per request. That is the number customers saw and could build on.

The problem was on the GPU side. Large language model servers keep a so called KV cache in GPU memory. It stores the intermediate attention data for the tokens of a prompt, so the model does not have to recompute everything for each new word it writes. On the GPUs that process prompts for GLM 5.3, that cache pool only had room for 681,280 tokens. In other words, the service promised about 1.05 million tokens of room, but the hardware could only hold about 65% of that for a single request.

Cloudflare is open about the fact that it knew about this mismatch before launch. The team expected the inference engine to simply reject any prompt that did not fit, with an error. Their long context tests came back with successful HTTP status codes, so they assumed the engine handled the gap gracefully. As the report puts it, they validated response codes and speed at long contexts, but not whether the output actually made sense.

That assumption was wrong, because of a second problem inside SGLang, the open source inference engine Cloudflare uses to run the model.

The bug inside SGLang

Workers AI runs GLM 5.3 in what is called disaggregated mode. One pool of GPUs, the prefill side, reads the prompt and builds its KV cache. Then that cache is copied over a fast network link to a second pool of GPUs, the decode side, which generates the answer token by token. Splitting the two jobs is a common trick to get more speed and better use of expensive hardware.

The KV cache is organized in fixed size pages that are shared between requests. When a request finishes, its pages are not wiped. The next request that gets them simply overwrites them. That is normal and important for performance, but it means the engine has to track very carefully which request owns which page.

Here is where it broke. When a prompt was too long for the pool, SGLang's scheduler marked it as rejected and swapped the prompt for a tiny one token placeholder, so the request could drain through the system without using real memory. On a single machine that works fine: the placeholder runs and the client gets an error.

In disaggregated mode, though, the admission checks on both the prefill and the decode side only asked whether a request was finished, not whether it had been rejected. A rejected request still looked alive. So the following happened, every single time a prompt crossed the threshold:

• The decode side was told how long the prompt was and reserved enough pages for it, for example eight.
• The prefill side admitted the rejected request anyway, ran its one token placeholder and wrote exactly one page.
• Prefill then reported that the transfer was complete.
• The decode side believed all eight pages were now filled with this request's data, and started generating.
• But seven of those eight pages still held whatever earlier requests had left there, including other users' prompts and partial answers.

The customer received an HTTP 200 instead of an HTTP 400. The token count in the response matched the long prompt, and the reply was coherent. The only visible clue was that it came back faster than a prompt of that size should take. And it could contain text from someone else's session.

That detail is what makes this kind of bug so uncomfortable. There was no crash, no warning and no garbled output that would make a developer suspicious. The system confidently answered using memory it did not own.

Who found it and how fast it was fixed

The issue was reported by security researcher Gadi Evron of Knostic through Cloudflare's HackerOne bug bounty program on September 8 at 23:11 UTC. He had noticed that very long prompts sometimes returned text from other users' sessions, and he included a reproduction script.

Cloudflare's response, according to its own timeline, went like this. Verification began on the morning of September 9. A security incident was declared at 13:22 UTC, and the product security team reproduced the leak with two independent accounts within 25 minutes. At 14:00 UTC the model was set to private, so all requests returned an error. By 14:35 the root cause had been found, and by 14:52 all prefill nodes ran a corrected configuration. A fully rebuilt deployment with a patched engine was validated against the researcher's script, and the model went public again at 19:40 UTC the same day.

The configuration fix was simple: Cloudflare changed GLM 5.3 to match the setup it used for GLM 5.2, which freed enough GPU memory to grow the prefill pool to about 2.04 million tokens, roughly twice the context window. With that, any prompt that fits the context window also fits the pool, and the faulty rejection path is never reached. Cloudflare also patched the engine so that a rejected request is retired with its original error before any memory is allocated or any cache is transferred.

Was anyone's data actually abused?

Cloudflare says no evidence of deliberate exploitation was found. After excluding its own tests and the researcher's, the affected requests matched ordinary long context use in timing and volume, not someone hammering the bug on purpose. The company also says no other model on Workers AI was affected, because every other disaggregated deployment has KV pools larger than its context length.

That is reassuring, but it is worth being precise about what it means. "No deliberate exploitation" does not mean nobody saw anything they should not have. It means that 155 replies went to people who were most likely just using the model normally, and some of those replies contained fragments of other customers' data. Cloudflare does not say what kind of data that was, and in a service used for coding agents it could easily include source code, documents or internal notes. Anyone who sent very long prompts to GLM 5.3 on Workers AI between August 27 and September 9 has a reason to read the report closely.

It is also fair to note what went right. A researcher reported it responsibly, Cloudflare took the model offline within about 40 minutes of declaring an incident, fixed it the same day, and then waited for upstream fixes before publishing a report that names its own mistakes. That is how disclosure should work.

This was not only a Cloudflare problem

The most important part of the report, at least for anyone running models themselves, is that the engine bug was not unique to Cloudflare. The same behavior had been reported to SGLang by another user as issue #30233 on July 6, 2026, almost two months before GLM 5.3 even launched on Workers AI. Cloudflare says it noticed that issue only while investigating its own incident.

SGLang is widely used to serve open models, and disaggregated prefill and decode is becoming a standard way to run them at scale. The fix first shipped in SGLang v0.5.20, with follow ups in v0.5.21, released October 2, and v0.5.22, released October 19. The SGLang maintainers at RadixArk published their own post as part of the coordinated disclosure.

Cloudflare also describes this as one example of a whole class of memory safety bugs in KV cache handling that it has been chasing for months. In some cases one part of the system frees a page while another part is still using it. In others, a request is aborted while the prefill side is still writing its cache over the network, and those late writes land in a page that has already been handed to someone else. In all of these cases the outcome is the same: corrupted output, or one user's data in another user's response, with no error raised.

If you run SGLang in disaggregated mode yourself, Cloudflare's advice boils down to a few checks:

• Measure the KV pool on every prefill and decode node from the engine's startup log and compare it with your context length, after every change to weights, quantization, parallelism or memory settings. If the pool is smaller, lower the context length.
• Upgrade to SGLang v0.5.22 or later, so that an over length request is rejected before any handoff.
• Keep deferred KV release enabled on both roles, and limit transfer batch sizes so a single timed out call cannot leave many writes in flight.
• Turn on the new KV checksum and KV canary features where your model supports them, and check your logs to make sure requests are really being verified.
• Configure your proxy so it does not abort requests unnecessarily, because every abort requires delicate coordination between machines.

What Cloudflare wants to change next

The report does not stop at patches. Cloudflare argues that configuration checks and lifecycle fixes reduce the number of ways a page can be misused, but they cannot make misuse impossible. The next bug, it writes, "will have a shape we have not seen."

Its proposed answer has two parts. The first is integrity by construction: every KV page would carry a tag that changes whenever the page is handed to a new request, and the engine would check that tag before reading. If a request tries to read a page it does not own, only that request fails, instead of returning someone else's data. Cloudflare says a prototype of this cost under 1% in throughput and latency, but was disabled shortly after testing on GLM 5.2 because of reliability issues, and was tied to specific architectures. A per transfer checksum that landed in SGLang is described as the first step in that direction.

The second part is more ambitious. Cloudflare wants to use formal verification, mathematical proofs of correctness, for the protocol that moves KV cache between machines over RDMA. The goal is to prove, as far as possible, that a bug like this one cannot happen again. The company itself admits that the provers will leave gaps, but argues that proving the rest makes those gaps easier to watch.

Why this matters beyond one model

It is tempting to file this away as a niche bug in one model on one platform. I think that would miss the point. As AI providers race to offer bigger context windows and lower prices, they squeeze more users onto shared GPU memory and split work across more machines. Every one of those optimizations adds state that has to be tracked perfectly. When that tracking fails, the failure does not look like an outage. It looks like a normal, helpful answer.

For businesses, the lesson is practical. Tenant isolation in hosted AI is not a property of the model, it is a property of the serving stack, and that stack is often built on fast moving open source projects. Asking a provider which inference engine it runs, how it isolates customers in memory and how fast it applies upstream security fixes is now a reasonable part of due diligence. Treating very long prompts with sensitive content as higher risk is reasonable too.

For the open source ecosystem, the story is more encouraging than it first sounds. A researcher found the problem, a large provider owned its configuration mistake in public, and fixes were pushed upstream so that everyone running SGLang benefits, not only Cloudflare's customers. The uncomfortable part is the gap between July 6, when the bug was first reported upstream, and the moment it was fixed. Bugs like this one need to be treated as security issues from the first report, not as edge cases.

Sources

• Cloudflare blog, Kevin Flansburg and Michelle Chen: Cross request KV cache exposure in the GLM 5.3 Workers AI model, and our remediation with SGLang, October 26, 2026
• SGLang: v0.5.22 release notes on GitHub
• Gadi Evron on Infosec Exchange about his report
• Earlier SGLang bug report: issue #30233