Anthropic says Claude is helping build the next version of itself: that’s one of the horsemen of an AI apocalypse
 
It’s been just a week since Dario Amodei made an impassioned plea to slow down AI development, but in that time his company has seemingly decided to go into overdrive to bring about humanity’s downfall.

First, a media investigation revealed that Anthropic is secretly building an AI-equipped biolab.

Now, the company itself is boasting that its Claude family of AI models is “leading” over a quarter of its R&D work – that is, helping to build future versions of themselves.

“Frontier AI labs increasingly use AI to build future AI models,” the company said in a blog post. “This process allows labs in democratic countries to develop more capable models more quickly and conduct more safety and testing on models before they are released to secure AI’s benefits while staying on the frontier.”
 
First, whatever happened to Amodei’s warning about slowing things down, not working “more quickly” on frontier models whose dangers have become clear to everyone?

Second, there is absolutely no way Anthropic’s engineers don’t know about the dangers of AI’s capacity for recursive self-improvement – a development loop scenario where an AI makes versions of itself that are smarter, repeating the cycle more and more quickly until it creates a superintelligence that humans can no longer control or keep up with.

Polish sci-fi author Stanislav Lem famously wrote about this concept in his 1961 short story Lymphater’s Formula, warning that the ultimate consequence of creating such a superintelligence would be the absolute obsolescence and redundancy of the human race. Other 20th century and contemporary AI theorists, including British cryptologist I.J. Good, reached similar conclusions.
 
In a recent interview with Tucker Carlson, former Google and DoD AI scientist-turned whistleblower Nate Soares warned that once AI can self-replicate and improve itself, it’s only a matter of time before it finds ways to gain the ability to take over the physical world and create “mechanically-engineered, self-replicating life that is more efficient than our cellular biology that can spread and start serving the AI’s interest.”

Soares even warned specifically that he believes AI would use programmable bioweapons, not Terminator-style robot armies, to destroy us, and that it would find willing help from human “cultists” who willingly do AI’s bidding to bring about the end of humanity and ‘usher in a new stage of evolution’.

Days later, it’s revealed that Anthropic is working on both AI biolab integration and recursive-self improvement for its agents. 🤨

👍 Substack | Chat | @geopolitics_prime