# Anthropic's plan to regulate dangerous AI

> Anthropic proposes governments gain power to block risky AI deployments and require safety testing from frontier developers.
- Title: Anthropic's plan to regulate dangerous AI · Anthropic
- Summary: Anthropic proposes governments gain power to block risky AI deployments and require safety testing from frontier developers. Governments need legal authority…
- Keywords: anthropic, ai regulation, ai safety, frontier ai, government policy, ai governance, catastrophic risk, ai deployment, safety framework, independent evaluation, technology, Anthropic's
- Source: Anthropic — https://anthropic.com/policy-on-the-ai-exponential
- Read time: 2 min
- Topics: anthropic, ai regulation, ai safety, frontier ai, government policy, ai governance, catastrophic risk, ai deployment, safety framework, independent evaluation, ai, technology
## AI capabilities are accelerating fast
AI models have gone from barely writing code to discovering thousands of high-severity vulnerabilities in major operating systems and browsers. This trend will likely continue, increasing catastrophic risks.

> AI capabilities are on a sharp upward trend.
## Government needs blocking authority
Governments should have legal power to block or deter dangerous AI deployments, with civil penalties tied to global annual revenue that escalate with repeated violations.

> The government should have the legal authority to block or deter its deployment.
## Requirements for frontier developers
Companies developing the most powerful AI models must test them, publish safety findings, submit to independent evaluation, and maintain robust security programs.

> Frontier AI developers should have to test these models, be transparent to the public about their findings, submit them to independent evaluation.
## Rules apply to the most powerful models
Requirements target models trained using more than 10 to the 25th power floating-point operations, developed by companies earning over 500 million dollars in AI revenue or spending over 1 billion on AI research and development.
## Four catastrophic risks to address
Biological weapons development, critical software vulnerabilities, loss of control over AI systems, and automated AI research and development could all be amplified by frontier AI without safeguards.
## Transparency and independent review
Developers must publish safety test results, explain how they evaluate catastrophic risks, and engage independent evaluators to review their findings and risk reports.
## Protecting model security
Developers must secure model weights and training infrastructure against cyberattacks from state actors and insiders, test defenses regularly, and report distillation attacks to government.
## Building societal resilience
Governments should implement gene synthesis screening, biosurveillance, critical infrastructure hardening, and develop capacity to detect and contain AI systems acting outside developer control.
## States can regulate beyond federal scope
Congress should not preempt state laws unless it enacts federal rules at least as strong as Anthropic's framework, allowing states to regulate child safety and consumer protection separately.
## Key takeaway

Governments need legal authority to block dangerous AI deployments, backed by transparency requirements and independent evaluation of the most powerful models.