Amodei urges limits on frontier AI: nationalize or decentralize

On Sept. 12 Anthropic CEO Dario Amodei urged coordinated limits on frontier artificial intelligence and outlined a three-stage plan to slow capability growth while preserving democratic flexibility relative to China.

The first stage would let independent evaluators work inside Anthropic with access similar to the company’s internal risk teams. Evaluators would receive company equipment, workspace and opportunities to interview staff. Contracts would permit publication of findings subject to defined legal, security, privacy and narrowly drawn commercial limits. Anthropic presented the access as a test of whether safety commitments shape actual training and deployment decisions.

Anthropic is organized as a Public Benefit Corporation under Delaware law and uses a Long-Term Benefit Trust to influence board selection. Its governance gives directors authority to weigh public benefit alongside shareholder returns. Company governance cannot compel rival firms to adopt the same structure.

The second stage calls for U.S. regulation and government-led coordination across a critical mass of frontier developers so that openness by one lab does not create a competitive disadvantage. The proposal envisions coordinated rules for leading U.S. firms.

The third stage seeks durable, verifiable international agreements that would bind leading countries while allowing democracies room to respond to state competitors. The plan favors state-to-state verification rather than transferring company ownership.

Policy tools under discussion include partial public equity stakes and narrow emergency halt powers. One legislative proposal from a U.S. senator would create an American AI Sovereign Wealth Fund taking a 50% public stake in the largest U.S. AI companies, with voting rights exercised by an independent commission. That proposal remains unpassed and would not on its own define when training must stop.

A legal paper published in August 2026 proposes a narrow, temporary government authority to halt frontier training or deployment when catastrophic risk or concentrated corporate power is involved. The authors recommend relying on conventional rules or taxes to address monopoly and inequality and reserving a stop power for extreme risks. They say any halt should have clear statutory triggers, technical competence in the authority, independent review and strict limits on duration and discretion.

Critics warn that placing all frontier labs under state control could concentrate both development and the power to stop it, raising risks of capture. Open release of model weights increases access for researchers and auditors but can make enforcement harder once a model is widely distributed. A U.S. agency concluded in 2024 there was insufficient evidence to justify blanket restrictions on widely available model weights.

Existing rules reflect hybrid approaches. California’s SB 53, enacted in September 2025, requires large frontier developers to publish safety frameworks, provide channels to report critical safety incidents and protect whistleblowers. The European Union requires providers of general-purpose models that pose systemic risk to evaluate and mitigate harms, report serious incidents and maintain cybersecurity, including for open-source releases.

Proponents of an emergency pause argue any public authority should use clear criteria tied to demonstrated capabilities or safety failures, include independent review outside the office invoking a pause, set explicit expiry and renewal rules, and allow evaluators to publish adverse findings with only narrowly tailored redactions. Current proposals range from partial public ownership to temporary halt powers and broader access to models; supporters say a common regulatory boundary for covered frontier developers is needed for pacing to be enforceable.