Anthropic proposes safety bill and benefit fund amid IPO scrutiny

Anthropic announced the creation of a Public Benefit Fund, a wholly owned LLC initially committed to 5% of the company's shares, with the potential to reach 20% as valuations increase. Co-founders have pledged to donate 80% of their equity to the fund. Simultaneously, the company circulated a confidential draft AI safety bill to lawmakers. The legislation would permit the government to block the release of a model if it poses a foreseeable imminent risk of causing more than $1 billion in damages or killing more than 50 people via bioweapons, loss of control, or cyberattacks. It requires independent evaluators registered with the Commerce Department to assess safety protocols, with blocking decisions adjudicated in federal court. The bill expressly prohibits other agencies from suspending model development, a provision critics view as limiting executive branch power.

David Sachs criticized the bill's definition of 'frontier' as 10^25 FLOPs, arguing it is outdated compared to the current 10^27 FLOPs standard. Critics including Nate Suarez and Jeffrey Ledish argued the bill makes it harder to restrict dangerous AI and removes state-level regulatory powers. In a separate development, New York City Comptroller Mark Levine urged Anthropic to pause its IPO to address safety concerns. Writing as investment advisor for the $325 billion NYC pension fund, Levine cited 80 pages of risk disclosures in Anthropic's IPO filings, which he described as unprecedented. He expressed concern over proposed super voting shares for the founding group, arguing that a gap between control and accountability is dangerous. Levine stated that if a catastrophic safety incident occurred after the IPO, it would be terrible for the stock, and that it is better to take time now to establish proper governance. Anthropic has opened discussions with his office following the letter.