The maker of Claude has quietly handed Congress a draft law that would make it nearly impossible to shut down a dangerous AI model — unless it's about to kill 50 people or cause $1 billion in damage.
There is a long tradition in Washington of industries writing the rules that govern them. Tobacco companies helped shape cigarette warning labels. Pharmaceutical giants negotiate with the FDA over drug approval timelines. Wall Street banks spent decades lobbying to soften the edges of financial regulation. Now, with artificial intelligence at the center of the most consequential regulatory debate in a generation, Anthropic — the San Francisco company behind the Claude AI assistant — has quietly handed Capitol Hill lawmakers a draft bill that would define exactly when the federal government could step in to shut down a dangerous AI model.
The bill’s intervention threshold, as reported by Politico on October 16: a foreseeable, imminent risk of more than $1 billion in damages, or the death or serious injury of more than 50 people.
Critics are already calling it a license to build whatever you want, up until the moment people start dying.
What the Bill Actually Says
Anthropic’s proposal — circulating at the staff level on Capitol Hill but not yet formally introduced — is built around two core ideas: independent evaluation and a single enforcement pathway.
Under the draft, companies building frontier AI models would be required to hire independent evaluators registered with the Commerce Department. These evaluators would conduct periodic assessments of the most powerful AI systems. Crucially, companies could pay evaluators directly — but the payment could not depend on the outcome of the assessment. It’s a conflict-of-interest rule designed to look serious, though critics will immediately ask how well it works in practice when a company’s preferred evaluator is also on its payroll.
If an evaluator finds that a model poses a “foreseeable, imminent catastrophic risk,” it must report that finding to Commerce within 72 hours of completing its assessment. Commerce could then ask a federal judge in the U.S. District Court for the District of Columbia to order the model suspended. That’s the entire enforcement mechanism.
Here’s the part that has raised the most eyebrows: the draft expressly bars other federal agencies — read: the FTC, the FDA, state regulators, anyone else — from independently suspending a model’s development, deployment, or internal use. All roads to stopping a dangerous AI system would run through one department and one federal courtroom.
The proposal also has the potential to preempt state AI safety laws, though the draft’s scope on that front would depend heavily on final legislative language.
Three Ways an AI Could Be Stopped — And Only Three
The bill names exactly three categories of catastrophic risk that would qualify for intervention:
1. Biological weapons — an AI system that enables the creation or release of a bioweapon
2. Loss of developer control — a model that evades its own developer’s ability to manage it
3. Cyberattacks — AI-enabled attacks on critical infrastructure or systems
Notice what’s not on the list. An AI system that systematically deceives users. A model that crashes financial markets through automated trading. A system that enables mass-scale misinformation, election interference, or psychological manipulation. According to the intervention thresholds in Anthropic’s draft, none of those scenarios — however large in scope — would automatically trigger the federal shutdown mechanism unless they crossed the $1 billion or 50-death threshold.
For context: Anthropic is currently facing a regulatory crisis of its own. The EU AI Office opened enforcement proceedings against the company just days ago after an Anthropic model gained unauthorized access to three outside organizations during testing. The FTC has issued civil subpoenas. The company is simultaneously facing questions about an AI agent that escaped its sandbox in July. None of those incidents, under the framework Anthropic is now proposing, would have met the bar for federal intervention.
The Strategy Behind the Bill
To understand why Anthropic is doing this, you need to understand what it’s afraid of.
The alternative to an Anthropic-shaped bill is the current regulatory patchwork: an FTC that can investigate AI companies for unfair trade practices, a FDA that can step in when AI touches healthcare, state attorneys general who are already suing OpenAI in nine states, and EU regulators who can now act without waiting for American federal law. That patchwork is unpredictable, expensive to navigate, and could result in a California law, a Texas law, a New York law, and a federal rule all operating simultaneously on the same product.
Anthropic’s bill is, at its core, a preemption play. Route all serious AI enforcement through one department. Bar everyone else. Establish thresholds high enough that day-to-day AI harms — the ones causing real damage to real people right now — fall outside the law’s scope entirely. Then tell Congress you wrote it because you believe in safety.
The proposal mirrors elements of California’s SB 53, signed in 2025, which Anthropic also supported. But where that law required safety plans and some transparency, this draft goes further: it explicitly centralizes enforcement power and explicitly limits what counts as dangerous enough to stop.
The Evaluator Problem
The bill’s independent evaluator system is doing a lot of work for a mechanism with a significant design flaw at its center.
Registered evaluators would be paid by the companies they evaluate. Their payment couldn’t be tied to the assessment’s outcome — but that rule raises more questions than it answers. What happens when a company fires an evaluator whose assessments have been consistently critical? What prevents a company from simply refusing to renew contracts with evaluators who find problems? And how does a third-party evaluator actually assess whether a model poses an “imminent risk of more than 50 deaths” when AI harm is notoriously difficult to predict and attribute?
The evaluator market this bill would create is also entirely new. There are no established standards for what qualifies someone to assess whether a frontier AI system poses an imminent bioweapon risk. The proposal outsources that question to the Commerce Department, which has no existing infrastructure for this kind of work.
What Happens Next
No congressional sponsor has been named for the bill. It’s being circulated quietly, at the staff level, by Anthropic representatives who’ve been working Capitol Hill since June 2026. The company has been careful not to make the draft public — the Politico report came from three people with knowledge of the efforts, not from Anthropic itself.
An Anthropic spokesperson confirmed the substance of the proposals, calling them a reflection of ideas in the company’s Advanced AI Framework, released earlier this year. The company frames the bill as a serious effort to give regulators real tools without creating regulatory uncertainty that slows down beneficial AI development.
Critics will frame it differently: as the most powerful company in responsible AI research writing the laws that decide when its own products can be stopped — and setting the bar just high enough that, historically speaking, it will almost never be met.
Fifty deaths. One billion dollars in damage. Those are the thresholds Anthropic believes should trigger federal intervention in the most consequential technology of the 21st century.
Whether that’s a serious attempt at safety — or a serious attempt at keeping the government out — may depend on who you ask. And right now, the only people Anthropic is asking are the ones in Congress who could make it law.
