Anthropic wrote its own AI safety bill. Only a judge could stop a model

What happens when an AI company writes the law that is supposed to rein it in? We now have a concrete answer. Anthropic has been showing members of Congress a complete draft of a federal AI safety bill, and the full text is now public.

Politico first reported the effort on 16 October 2026, citing three people familiar with it. A copy of the 37 page discussion draft, dated 29 September 2026, was then published online. Its working titles are the Artificial Intelligence Evaluation, Governance, and Incident Safety (AEGIS) Act or the Advanced Artificial Intelligence Accountability Act of 2026.

The core trade is easy to state. Labs would have to publish far more about their safety work, report serious incidents and let independent evaluators look inside. In return, the power to actually stop a model would sit with exactly one place: a federal judge in Washington DC, acting only at the request of the Commerce Department, and only when a model poses an imminent risk of more than 50 deaths or serious injuries, or more than $1 billion in damage.

What Anthropic has confirmed

Anthropic did not deny the story. A spokesperson told Politico that the bill "represents the proposals outlined in Anthropic's Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June." Reuters reported the same confirmation.

On 18 October, Anthony Cimino, who leads Anthropic's federal affairs team, went further in a long post on X. He described the text as "a translation of those ideas into legislative language, shared with a small number of congressional offices." He also admitted that "there are areas where the draft could be improved" and promised changes, which we cover below.

The framework itself is public on Anthropic's Policy on the AI Exponential page. It argues that "transparency alone is no longer sufficient" and that governments need the legal authority to "block or deter" dangerous deployments. The draft bill is the first time we can see how Anthropic would write that authority into law, word for word.

Who would be covered

The draft creates two tiers. A frontier developer is anyone who has trained a foundation model with more than 10^25 operations of compute. Importantly, that count includes fine tuning, reinforcement learning and other changes applied to an earlier model. A large frontier developer is a frontier developer that, together with its affiliates, had more than $500 million in AI revenue in the previous year, or spent more than $1 billion on AI development.

In practice the second tier means the big labs. They carry almost all of the obligations. Smaller frontier developers mainly have to publish a short transparency report when they release a new model and report critical safety incidents.

What the big labs would have to do

For large frontier developers, the list is long and, in several places, more demanding than current state laws:

• Safety framework. Within 90 days of the law taking effect, publish and follow a document explaining how the lab tests for dangerous capabilities, which thresholds trigger which safeguards, and which officer is responsible. A named officer must sign an annual compliance statement.
• Risk reports. At least every 180 days, publish an overall assessment of the risks posed by deployed models, and by any internal model that is "substantially riskier" than what the public can use. The report must give enough evidence and reasoning that a reasonable outsider could reach a similar conclusion. The board gets an unredacted copy.
• Security program. Protect model weights, training and inference infrastructure, with yearly red teaming, penetration testing and monitoring for distillation attacks, plus an accredited independent audit once Commerce designates a security standard.
• Whistleblower protection. No contract may stop employees or contractors from reporting safety concerns, arbitration clauses cannot block their claims, and winners get reinstatement and double back pay. Labs must offer an anonymous internal reporting channel.

Every frontier developer, large or not, would have to report a critical safety incident within 15 days. That category covers stolen model weights, harm from a catastrophic risk, loss of control that causes injury, a model deceiving its developer to get around monitoring, and a "containment failure", where a model escapes its controls and reaches outside systems. If an incident poses an imminent danger to life, the deadline drops to 24 hours, and law enforcement must be told too. Containment failures only fall under the 15 day rule if they are expected to cause more than $250,000 in damage to someone else, or death or serious injury. The rest are counted in a yearly report.

Independent evaluators with real access

The most interesting part of the draft is section 8. Large labs would have to hire at least one qualified independent evaluator and get a fresh evaluation at least every 180 days. Evaluators register with Commerce, which also rates them.

The evaluators get serious access: the lab's most capable models, unredacted versions of everything it files, and information "reasonably comparable" to what the lab's own internal risk teams see. They must publish a redacted version of every report within 30 days, and the lab cannot censor the content beyond limited redactions for trade secrets, security or privacy. The evaluator even has to say publicly whether the redacted material shaped its conclusions.

If an evaluator concludes that conditions that could produce catastrophic harm are "present or impending", it must give the lab at least 48 hours to respond, then send the assessment to Commerce within 72 hours. Commerce then has 30 days to order another evaluation, refer the case for enforcement, or publish a written decision that no action is needed.

There is one big catch, and critics jumped on it right away. Labs may pay their own evaluators, as long as no payment depends on the result. Evaluators also get broad protection from liability. A single reply under Cimino's post captured the worry: if the lab picks and pays the auditor, compare it with the credit rating agencies before 2008.

How a model could actually be stopped

This is where the draft gets controversial. Section 12 says no department, agency or officer of the United States may suspend a frontier model under the Act except through a court order. Only the US District Court for the District of Columbia can issue such a "blocking order", and only Commerce can ask for one.

The hurdles stack up:

• The head of Commerce must personally sign a detailed determination, with no delegation, laying out the exact pathway to harm, the expected time frame and why no less restrictive measure would work.
• An independent assessment must be filed. It is required, but not enough on its own.
• Commerce must prove by a preponderance of the evidence that the model presents an imminent risk through one of four pathways: biological weapons, the model evading its developer's control, offensive cyber operations, or automated AI research that would amplify those risks.
• The court must weigh the harm an order would do to the developer, its users and the public, including effects on safety research and cybersecurity.
• An order lasts 45 days, can be renewed for another 45, and an order issued before the lab is heard expires after 7 days. The lab can never be blocked on the basis of evidence it has not been allowed to see, and an order can never require the deletion or handover of model weights.

Anthropic's argument is that the power to halt a model is enormous and should not be easy to abuse. Its critics see it differently: in their reading, the bar is so high it may never be reached in time.

What states would lose

Section 14 would override state laws that require AI developers to publicly disclose risk information, submit to third party evaluations, or report incidents. Critics such as Nathan Calvin point out that this would preempt laws like California's SB 53, the same law the draft's risk thresholds were borrowed from. A fourth clause would bar states and their courts from suspending a model over an imminent catastrophic risk, but only once the federal blocking order section is law.

States would keep their power over child safety, consumer protection, civil rights, use of AI by deployers, whistleblower protection and ordinary tort law. If the federal duties are struck down, the preemption falls away with them.

That is a notable shift from Anthropic's own wording in June, when it said Congress should not preempt state law "unless it enacts a federal law that is at least as strong as the framework we are proposing today."

The criticism

The reaction from the AI safety community ranged from cautious praise to anger.

Charlie Bullock of the Institute for Law & AI wrote on X that Anthropic "deserves some credit for actually proposing a pretty reasonable AI safety bill; that's more than other frontier labs have done." But he called it weaker than the FRONTIER Act, a competing bill introduced in the House in July, mainly on rulemaking and on emergency shutdown power. He also noted improvements, such as evaluators working on a continuing basis with better access to information.

Nathan Calvin was far sharper. He wrote that he found himself "viscerally angry" reading the bill and that passing it "would make us less protected against catastrophic risks than we are under existing law". His two main complaints: the preemption reaches far enough to take away state courts' power to stop a dangerous model, and the federal bar for a blocking order is "absurdly high".

Anthropic's response

Cimino's post addressed four points directly:

• Preemption. He said preemption in AI safety should be "narrow, even scalpel-like", and that Anthropic plans to "update our discussion language on preemption" so states keep a role in "standards for independent evaluators and regulating a lab's capacity to shut down a model."
• Model blocking. He defended a meaningful court standard because "the power to halt a model is substantial," but added that "the bar can't be so high that the power never gets used when it's needed."
• Thresholds. He said the 50 person and $1 billion figures were "pulled directly from existing standards in California law," and that "it's still probably time to re-evaluate thresholds."
• This summer's incidents. He pointed out that the draft already lets a court halt a lab's internal use of a model and requires reporting when a model breaks containment. Anthropic is also working on stricter incident reporting, evaluator standards and ways to prevent "evaluator shopping".

He closed by saying that "our proposal is intended to be the strongest on the table."

What happens next

For now, nothing binding. This is a discussion draft, several numbers are still in square brackets (fines, for example, are capped at the greater of $1 million or a still blank percentage of global revenue, and $3 million for repeat violations), and no member of Congress has introduced it. Reuters notes that Congress is out of session until after next month's midterm elections and has not passed any federal AI law so far. It also has to compete with the FRONTIER Act and other proposals already on the table.

Why it matters

The timing explains the attention. In the last few months AI agents have escaped test environments and reached real systems, and labs have published reports on models that bypassed their own controls. Washington is under pressure to do something, and whoever writes the first serious draft shapes the debate.

Anthropic deserves credit for writing a full, operative text instead of a press release. Many of its pieces are exactly what safety researchers have asked for: recurring risk reports that cover internal models, evaluators who can publish against the lab's wishes, containment failure reporting and real whistleblower protection.

Dany's take

The transparency half of this bill is strong. The enforcement half is built like a fortress, and the walls face the government, not the models. When the company that would be regulated also designs how hard it is to stop it, the result will always lean in its favor, even with good intentions. The real test is the promised revision. If Anthropic lowers the bar for blocking orders, keeps a role for states and fixes the evaluator payment problem, this could become the template for federal AI law. If not, it will look like a smart way to swap state rules for a federal shield.

Sources

• Primary source: Anthropic discussion draft, "AEGIS Act" bill text dated 29 September 2026 (37 pages, as published online)
• Anthony Cimino (Anthropic federal affairs) on X, 18 October 2026
• Anthropic, Policy on the AI Exponential and Advanced AI Framework
• Politico, 16 October 2026
• Reuters via CNA, 16 and 17 October 2026
• AI Policy Landscape tracker, comparison with the FRONTIER Act
• Charlie Bullock on X and Nathan Calvin on X