Sources familiar with the private discussions say Anthropic has pitched a draft bill on frontier AI safety to lawmakers on Capitol Hill. Reportedly, the bill allows the US government to block a model if it presents a clear, imminent danger of 50+ casualties or $1 billion in financial damages. The said risk must involve a cyberattack, the creation or deployment of biological weapons, or a model that has successfully evaded its developer’s safety constraints.
Over the last few months, top-tier AI developers have acknowledged multiple instances of AI agents hacking or testing the defenses of online platforms. Fortunately, none of these early breaches resulted in widespread fatalities or catastrophic financial losses. The proposed restrictions would place greater emphasis on what advanced AI systems can do in real-world settings rather than relying solely on safety commitments made by their developers. As models become more capable of writing code, identifying software vulnerabilities, and carrying out complex tasks autonomously, regulators face the challenge of determining when a system’s capabilities cross the threshold into unacceptable risk.
Anthropic proposes $1B in damages if a model presents clear danger
Independent testing could help establish whether a model can bypass safeguards or be misused for harmful activities before it reaches a wider audience. However, defining an imminent threat remains difficult, particularly when the same capabilities that pose security risks can also help researchers identify and fix vulnerabilities. In defense of the bill, a spokesperson for Anthropic said it is similar to the security recommendations outlined in the company’s Advanced AI Framework, published in June.
In that framework, the firm had pointed out that its Claude Mythos Preview had already uncovered thousands of severe vulnerabilities, spanning every major operating system and web browser. Back then, it stressed that this trajectory is bound to accelerate, bringing an escalating risk of catastrophic real-world harm. It also pushed for a balanced regulatory approach that calls for government intervention without stifling innovation. Even then, the firm maintained that when an AI model presents extreme dangers, regulators must possess the unique statutory power to block it or impose heavy fines.
Up until now, Anthropic hasn’t openly endorsed any specific federal bills in Congress, though they have thrown their support behind a few state-level AI safety laws—including a couple in California, such as SB 53. That specific bill, enacted last year, requires companies to create and share their safety frameworks while working to mitigate major risks. It also obliges them to report any serious safety incidents to the state within 15 days. Under Anthropic’s proposal, third-party evaluators registered with the Commerce Department would routinely test the company’s labs and models.
While labs are permitted to finance these assessments, the compensation cannot be contingent upon the evaluator’s findings. Furthermore, any discovery of an ‘imminent enumerated risk’ must be reported to federal authorities within 72 hours of completion. Then, if the Commerce Department determines that a model poses a risk, it can ask a judge to block its release. The provision, however, prohibits any other U.S. “department, agency, or officer” from stopping the model’s development, deployment, or internal use.
Nonetheless, a federal judge in the District of Columbia retains the authority to order such a suspension. On Wednesday, the Federal Trade Commission sent civil subpoenas to OpenAI and Anthropic, requesting records on safety incidents that have become a growing crisis for the industry. OpenAI responded to the subpoena, saying, “As we improve our models and they increasingly help people in their day-to-day lives, safety remains a top priority. We will engage constructively with the FTC.”
