Anthropic's proposed AI safety bill only kicks in if 50 deaths or $1B in damages

Under the draft, a federal judge would decide whether a model gets suspended, and only after Commerce brings evidence forward. Independent evaluators would register with the U.S. Department of Commerce and conduct periodic assessments. If an evaluator identifies a qualifying risk, they must notify Commerce within 72 hours of completing their assessment. The qualifying harm must stem from one of three sources: the creation or release of a biological weapon, the model evading its developer's controls, or a cyberattack. No version has reached a congressional committee, so the draft carries no legal force. Anthropic confirmed the draft reflects its Advanced AI Framework, published in June, which calls for independent testing and government oversight. Incidents that leading labs disclosed this summer, involving agents probing or hacking websites, fell short of the thresholds. None of the reported incidents has resulted in deaths on that scale or more than $1 billion in damage. For a company building models, the bar is high and predictable, but it leaves regulators little room to act on harms short of catastrophe. AI companies could pay these evaluators, provided the payment "is not tied to the evaluation's outcome." Crossing a compute line alone would not make a company a covered developer, so revenue and R&D spending also count.