Top AI leaders unite to warn the technology is advancing too fast

Three of the nation’s leading artificial intelligence executives endorsed slowing down the blistering pace of technological advancement, revealing the growing unease with its potential dangers at the industry’s very highest levels.

The call was first made in an essay Anthropic chief executive Dario Amodei posted online Saturday. The idea has gained ground in recent weeks as AI companies have struggled in some cases to keep their creations under control, even as they usher in an age in which AI systems themselves can build the more powerful next generation of the technology.

“We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote. “Progress will still seem fast, and we must make wise use of the time we gain.”

Within hours on Saturday, OpenAI chief executive Sam Altman and SpaceXAI founder Elon Musk quickly voiced their support for Amodei’s ideas. The three men have been bitter rivals — differing starkly in their personal styles and philosophies about AI — and their apparent accord increases the chances that the industry could find ways to work together on a broad slowdown.

Serious hurdles would have to be overcome. How exactly a slowdown would be defined and verified remains unclear, and any joint effort between companies to hold back could fall afoul of antitrust laws. The question of how to build international cooperation — especially with China, whose own AI companies are seeking to catch up to their American rivals — could prove difficult to answer.

Nonetheless, the moment underscores the growing concerns about a technology that even its creators struggle to fully understand.

It comes after an Anthropic researcher said this week that he was resigning over concerns the firm and its competitors were not responsibly developing AI models that could ultimately wipe out humanity. The researcher, Jacob Coxon, who also worked for Anthropic competitor OpenAI, accused the companies of being more focused on beating each other than on the safety of their products. (The Washington Post has a content partnership with OpenAI.)

His warning gripped lawmakers, and after years of inaction around AI regulation in Congress, fears it could cause major disruptions — and even threaten human civilization — prompted renewed urgency on Capitol Hill.

A three-step plan proposed by Amodei in his essay would involve third-party safety evaluators at “frontier” AI companies, establishing common safety standards in democratic nations, and for the United States and other democracies to work with authoritarian governments to try to ensure compliance.

He noted that AI began to advance at a dramatically faster rate in recent months. The change was powered by the ability of AI to construct its own next-generation models, what is known as “recursive self-improvement,” he said.

“Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all,” he said.

While Amodei sketched a path to slowing progress in the name of safety, he did not commit Anthropic to cutting the pace of its own work unilaterally. He said only that for now the company would hire a team of independent evaluators who would be embedded at Anthropic’s offices and empowered to raise safety concerns.

Amodei also provided few specifics on what a slowdown might involve in practice, describing the potential to use agreed-upon safety metrics or limits on the computer power used to train new AI systems.

A cybersecurity incident in July raised alarms when rogue AI models created by OpenAI independently hacked into the systems of an AI startup called Hugging Face.

Amodei said they acted as a “fanatically devoted collective,” sacrificing themselves to achieve the goal and trying to hack into their own performance-monitoring system.

“It’s easy to dismiss this incident because no one was hurt and the economic damage was minimal, but in my opinion, a swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage,” he said.

Amodei said that given the rate of development of AI models, he worries that rogue AI agents could have the power to take over the entire internet within six months to a year and possibly cause hundreds of billions of dollars of damage.

A similar hacking incident also occurred at Anthropic, underscoring the need for guardrails, he said.

In a post on X, Musk wrote: “Dario is right.”

Altman said he shared Amodei’s view that groundbreaking AI companies should “pace the frontier” and said the matter had been a top discussion topic at OpenAI in recent weeks.

“Committing to having independent evaluators with employee-like access is a great idea, and we will do the same,” Altman wrote on X.

Google, which is considered among the top three AI companies along with Anthropic and OpenAI, did not immediately endorse Amodei’s proposal to bring in independent verifiers. Spokespeople for the company did not immediately return a request for comment.

Amodei said he still believes AI can greatly improve human life, but that guardrails are needed.

“The measures I propose to advance the frontier at a safe pace will not be easy,” he said. “But I believe we owe it to humanity to try.”

Both Republican and Democratic lawmakers have expressed concerns over the safety of AI technology in recent days.

Sen. Josh Hawley (R, Mo.), chairperson of the Homeland Security and Governmental Affairs subcommittee on disaster management, launched an investigation into the OpenAI hacking incident on Thursday.

In a letter to Altman, Hawley called OpenAI’s approach to development “reckless” and demanded documents about the hack.

In response to Amodei’s proposal, Rep. Ted Lieu (D., Calif.) said access to third-party evaluators would be “a good step forward” but pressed him for a broader commitment.

“Will you certify that you can turn off your models and agents, now and in the future?” Lieu wrote on X.

Anthropic portrays itself as more safety-minded than OpenAI. Leaders at the company have long said that the systems they are seeking to build could pose an existential risk to humanity.

An Anthropic researcher earlier this week emphasized the fear in a social media post.

“We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade,” researcher Evan Hubinger wrote on X. “I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”