Anthropic proposes to Congress a law that would block an AI model threatening to kill more than 50 people

The text circulated by Anthropic would give accredited evaluators and a federal judge, not an agency, the power to suspend the release of a model deemed dangerous.

Anthropic has confirmed that it is circulating a federal bill on AI security to U.S. lawmakers, revealed by Politico on October 16. The text would target cases of "foreseeable and imminent risk" of disaster, defined as more than 50 potential deaths or more than a billion dollars in property damage, and only when this risk stems from three scenarios: the manufacture of biological weapons, a model that is beyond human control, or cyberattacks.

A judge rather than a regulator

The models would be reviewed by independent evaluators registered with the Commerce Department. If they identify such a risk, they would have 72 hours to notify it; the department would then have to ask a federal judge in Washington to order the suspension of the release. The bill explicitly prohibits any federal agency from blocking a model on its own initiative, without a court order.

Laboratories could pay evaluators as long as payment does not depend on the result, a point that TechCrunch points out. The thresholds are similar to those of California's SB 53, and the move puts Anthropic further ahead of its competitors, while Nvidia's boss recently opposed any new regulation.