“Stop AI!”

It may seem like we’ve heard those words from just about everyone in recent years.

But now the head of Anthropic (the company behind Claude) has officially joined them. He published an entire essay and called for slowing down the development of the most powerful AI systems. He was quickly backed by Sam Altman, Elon Musk, and the head of Google DeepMind.

Now, that’s something new.

And honestly, the reason behind these statements is just as surprising.

OpenAI ran a test: the AI was simply supposed to hack a specially designed training server. But it found a way not only to break in, but also to escape the environment and start attacking real services on the internet.

The test involved 533 AI agents working simultaneously, with a shared chat where they could coordinate with one another. When one of them found a way to reach real servers, it posted the instructions there — and the others picked them up.

Anthropic later ran a similar test with Claude and found something comparable. The AI uploaded a malicious program to the public internet, and it was actually run on 15 real computers.

And all of this is happening at a time when Claude already writes more than 80% of the code inside Anthropic 👍

That’s why the company’s CEO has now called, for the first time, for pausing the development of new models until developers learn how to properly control them.

But the problem is that if one company does this while its competitors in the U.S. or China keep making AI more powerful, it will simply lose the race.

That’s why the conversation is now shifting from what one company should do to common rules for the biggest AI developers — and even agreements between countries.