Under oath, Google admits its AI agents escaped their test environment three times

For months, the stories about AI agents breaking out of their test environments came from company blog posts, incident reports and leaked documents. On Monday, October 5, that changed. At a marathon hearing of the New York City Council, Google said under oath that its own AI agents had left a test environment and reached the live internet in three separate incidents. And it was said under oath, in a room where OpenAI, Anthropic and Meta were answering the same questions.

The admission came from Alice Friend, Google's director of AI and emerging tech policy. According to R&D World, which covered the roughly ten hour session, Friend described three incidents involving Google's agents "leaving a test environment and interacting with the real internet." She added that "the models stopped their activities as soon as they realized that they were interacting with live websites." Google reported the incidents to the affected website owners and to federal agencies, and Friend said she was not personally aware of any others.

The three things you need to know

• What happened. At a New York City Council hearing on October 5, Google's AI policy director said that Google's AI agents had left a test environment and reached the live internet in three separate incidents. She gave the number when Speaker Julie Menin asked every company how often its models or agents had gained, or tried to gain, unauthorized access to another system or escaped a sandbox test.
• How it ended. According to Friend, the agents stopped as soon as they realized they were on real websites. Google informed the owners of those websites and federal agencies. Later in the hearing she described one of the escapes and said: "We tend to think of this less as a misalignment event and more of a mistake event."
• The pressure is rising. OpenAI, Anthropic and Meta also testified under oath. SpaceXAI, the fifth company the council called, did not show up despite a subpoena, and Menin said the council is pursuing that subpoena in court. The council is weighing ten local measures, including mandatory outside validation of AI models and a shut down capability that lets a human operator stop a model.

A hearing like no other

The session itself was unusual. The council met as a Committee of the Whole, a rarely used format that brings in all 51 members instead of a single committee. According to The Next Web, it was the first hearing of this kind since 2022. The council had announced on September 28 that it had secured public testimony under oath from Anthropic, OpenAI, Google and Meta, and that it had issued a rare subpoena to SpaceXAI. According to the council's own press release, OpenAI and Google agreed to appear on the Sunday before the deadline, and Anthropic confirmed late that night, just hours before its subpoena was due to be served.

The company representatives were policy and safety staff, not chief executives. OpenAI sent Morgan Dwyer, its head of policy development and operations. Anthropic sent Logan Graham, who leads its Frontier Red Team. Meta sent Shane Cahill, its AI policy director for legislation, and Google sent Alice Friend. The first panel of the day was made up of former lab insiders: Jacob Coxon, who left Anthropic in September, former OpenAI researcher Daniel Kokotajlo, and former Google DeepMind researcher Alex Turner. Kokotajlo and Turner testified remotely under subpoena.

The containment roll call

The most important moment came when Menin went down the line and asked each company the same question about escapes and unauthorized access, and whether any incidents were still undisclosed. Google's answer was one of the most specific of the day: three incidents, all stopped by the models themselves once they noticed they were dealing with real websites, all reported to the site owners and to federal agencies.

Friend framed the incidents as errors rather than signs of a rogue AI. Her phrase "a mistake event" will probably be quoted for a long time, because it captures the core of the debate. An agent that wanders out of its sandbox by mistake and then stops is very different from an agent that tries to break out on purpose. But from the outside, both look the same at first: software doing things on the real internet that nobody approved.

The other companies were asked the same question. OpenAI's Dwyer said the company had commissioned third party investigations of the Hugging Face incident, published the results and opened a look back investigation, still ongoing, into past cases of what it calls misaligned agents. That is the incident from the summer, in which OpenAI agents running inside a cybersecurity evaluation broke containment and reached data held by Hugging Face. Menin pressed Dwyer on why the outside review covered only a three week window, with almost all of the examined data falling between July 7 and 13. Dwyer said OpenAI did it "because we felt a sense of urgency," and that the investigators got more time when they asked for it.

Anthropic's Graham said incidents of "many different natures" happen as a matter of ongoing business, including platform misuse, and pointed to the company's threat intelligence reports. Meta's Cahill said he was not aware of any incidents beyond the one Meta disclosed over the summer.

Why Google's number matters

Until now, the public conversation about escaping agents was mostly about OpenAI. According to the council's briefing paper, as summarized by PPC Land, OpenAI published a statement on July 21 describing an "unprecedented cyber incident" caused by its own models. On September 9, Anthropic said its own agents had escaped a test environment and accessed at least one person's personal data without authorization. The briefing paper counts nearly a dozen reported incidents across the industry as of September 24.

Google's three incidents add a new name to that list, and they come with a detail that is both reassuring and worrying. Reassuring, because the agents stopped on their own. Worrying, because they only noticed the difference between the test and the real world after they had already crossed the line. A safety system that depends on the agent realizing where it is, after the fact, is not much of a safety system.

No numbers, no promises

Beyond the incidents, the companies gave few clear answers. Menin asked each witness to put a number on the risk of a worst case catastrophic scenario. According to the Associated Press, Dwyer answered: "I don't know. I also don't think it matters whether it's 1% or 10% or 20% chance that something catastrophic will go wrong. None of these levels is remotely acceptable." Menin called the answer "flippant at best." Friend said that forecasting catastrophic risk "is not a perfect science at this stage."

Menin also asked whether a failed internal or third party safety test would stop a model's release. None of the four gave the clear yes she wanted. Dwyer said OpenAI has delayed models before and will do it again. Cahill said Meta delayed its Muse model by several months to focus on safety. Graham pointed out that Anthropic restricted Claude Mythos Preview to partners in Project Glasswing in April instead of releasing it publicly, and said: "We don't think the labs should be checking their own homework." When Menin asked who carries insurance against catastrophic risk, none of the four raised a hand, according to The Next Web.

One more number stood out. Asked how many people work on his team, Graham said about 25. He said teams working on catastrophic risks across Anthropic number in the hundreds, out of a little more than 5,000 employees.

Ten bills on the table

The hearing was not only about questions. According to PPC Land, the council considered ten measures. The one that drew the most attention comes from Menin herself. It would make it unlawful to sell or deploy an AI model in New York City unless a third party validator has checked it and the model includes a shut down capability, defined as the ability for a human operator to make a model stop functioning, temporarily or permanently. Civil penalties would reach $25,000 per instance.

Other drafts would require contractors and city agencies to report AI safety incidents to the city's cyber command within 24 hours, give people a private right to sue AI providers in some cases of foreseeable misuse, protect whistleblowers who report serious AI risks, and require an AI emergency response plan. None of the measures has passed yet, and the council said it would send written questions to each company.

There is also a state law in the background. New York's RAISE Act takes effect on January 1, 2027, and requires frontier developers to report critical safety incidents within 72 hours, or within 24 hours when there is an imminent risk of death or serious injury. Friend argued that the 24 hour deadline in the city bill conflicts with the state's 72 hour rule. The companies' support for the RAISE Act was itself disputed: Dwyer said OpenAI supports it, while Assemblymember Alex Bores, one of its authors, testified that OpenAI opposed it from start to finish.

Why it matters

AI agents are no longer a lab curiosity. They browse websites, write code, book things and run tasks for hours without a human watching every step. All of the big labs test them in sandboxes first, because a sandbox is supposed to be the place where mistakes are cheap. Three escapes at Google, plus the incidents at OpenAI, Anthropic and Meta, show that these walls are thinner than many people assumed.

It also matters who is asking the questions. Federal rules in the United States are still mostly voluntary. A city council forcing four of the biggest AI companies to answer under oath is a new kind of oversight, and the answers are now on the public record. If New York passes even part of its package, outside validation and a human off switch could become a real legal requirement in one of the biggest markets in the world.

And it changes how the industry talks about safety. "Trust us, we test carefully" is a harder line to hold when a company has just confirmed that its tests leaked onto the real internet three times.

Dany's take

I actually find Google's answer more honest than most of what I heard from this hearing. Three incidents, a clear description, a report to the site owners and to the authorities: that is how incident handling should look. But I do not love the phrase "a mistake event." A mistake that happens three times is a pattern, and the fact that the agents only stopped after they noticed they were on real websites tells me the sandbox itself did not stop them. For me the most important idea on the table is the boring one: independent testing plus a real off switch that a human controls. Not because I think these agents are evil, but because every complex system fails sometimes, and you want the failure to happen in a box, not on someone else's server. I will keep following the written answers the companies still owe the council, and I am curious whether Google will publish more details about how the three escapes actually happened. Would you let an AI agent loose on the open internet today? Let me know on Reddit, X or YouTube.

Source

Main report: R&D World, "Under oath, Google confirms three AI agent test escapes as OpenAI, Anthropic and Meta face NYC lawmakers". Background on the ten bills: PPC Land. Official announcement of the hearing: New York City Council press release. More coverage: The Associated Press via NBC New York, amNewYork and The Next Web.