What Dario is proposing is closely related:
"Thus, a key part of pacing within democracies is to keep democracies’ AI lead over autocracies as large as possible, to give us the breathing room we need in order to pace effectively."
In other words, Anthropic and OpenAI want to widen the "gap" between what their AI can do and what the rest of the world can do.
This doesn't necessarily mean that they will stop AI development and training.
What this leads to is:
- Anthropic, OpenAI, and other frontier labs will need to work together to make sure that every lab maintains alignment standards.
- These labs will continue using RSI to advance their internal models with "employee-like access".
- These models WON'T be released to the public until the "gap" is sufficient and until alignment standards are met.
What about China?
> "Distillation of frontier models allows lagging companies to narrow the gap using a fraction of the cost it would take to develop their own AI independently."
> "If we execute these measures well, I believe they would slow China’s progress enough to widen America’s lead significantly over the next 3–5 years — the window when AI becomes geopolitically most important."
The assumption behind these measures is simple: without being able to distill frontier models, it will take China significantly longer to close the gap with top-tier models.
All the above may fay fail. China may or may not take the lead in AI progress.
Yet, it's not a surprise that AI can already be used as a cybersecurity weapon. Note that the top point on the "frontier" line describes defenders' capabilities available to companies with Daybreak access and similar. However, the top point on the "open-weight" line is accessible to everyone.
Even with current levels of intelligence, we will start seeing more and more major security incidents around the globe.
> “Everything that makes it successful is exactly what makes it dangerous.”
We should be monitoring this very closely 👀