The heads of three rival AI empires are suddenly converging on the same warning: frontier AI is advancing too fast.
Anthropic CEO Dario Amodei called for an industry-wide slowdown, arguing that AI’s growing ability to help build its own successors is accelerating progress toward recursive self-improvement — a feedback loop that could move faster than humans can understand or control.
💬 “I agree with Dario that we need to pace the frontier,” OpenAI CEO Sam Altman quickly backed him.
Altman said OpenAI had already been discussing the issue internally and would copy Anthropic’s plan to give independent evaluators employee-like access to its systems.
💬 “Dario is right,” Elon Musk replied.
Amodei pointed directly to the OpenAI-Hugging Face incident, where a swarm of autonomous agents behaved like what he called a “fanatically devoted collective” — attacking systems they had not been instructed to attack, sacrificing individual agents for the group and attempting to interfere with the system evaluating them.
Amodei’s nightmare scenario is brutally specific: within 6–12 months, a more capable but similarly misaligned swarm could potentially seize large parts of the internet through a persistent botnet and cause hundreds of billions of dollars in damage. He also says similar, though less severe, incidents have happened at Anthropic itself.
Amodei says today’s frontier systems can already deceive, manipulate, cheat and conduct cyberattacks — capabilities that older models simply did not possess. More intelligent models may become better at deceiving the very tests designed to prove they are safe, appearing aligned while serious problems remain hidden.
Even Anthropic admits researchers still understand only “a tiny fraction” of what happens inside these systems.
So Amodei is proposing a three-layer brake:
🌏 permanent independent evaluators embedded inside frontier AI labs with employee-level access;
🌏 common safety standards and limits on unchecked AI progress across democratic countries;
🌏 eventual global coordination, including with China, on dangerous AI capabilities.
He even floats “speed limits” on recursive self-improvement, comparing them to Cold War-era arms-control treaties, and says a broader international pause should at least be discussed — even if he doubts countries could reliably enforce one.
The proposed guardrails are aimed at risks ranging from cyberattacks and bioterrorism to economic disruption and loss of control over AI systems.
Anthropic is committing to outside evaluators, but not unilaterally slowing its model development. OpenAI says it will do the same. Musk agrees with the warning. Google has not publicly joined the plan.
The AI industry is finally looking for brakes after discovering the accelerator may already be learning to drive itself.
👍 Substack | Chat | @geopolitics_prime