Security Experts Want the US and China to Promise Never to Let AI Control Nukes

Security experts from the United States and China have proposed rules that would bar artificial intelligence from independently launching cyberattacks against nuclear command systems or deciding to use nuclear weapons.

A recent report by Brookings and Shanghai-based Fudan University urges the U.S. and China to extend their commitment to human control of nuclear weapons to other military uses of AI, including cyberattacks on nuclear command systems.

“Presidents Donald Trump and Xi Jinping can lead in governing these dangers by agreeing that only humans, not AI, should make the decision to initiate a cyberattack against each other’s nuclear command, control, and communications systems (NC3) or critical infrastructure,” the researchers wrote.

NC3 systems help governments detect threats, decide whether to use nuclear weapons, and transmit launch orders, but failures at any stage could disrupt communications, misread an attack, encourage a first strike, or trigger an accidental or unauthorized launch.

“Extending the principle applied to nuclear weapons—that attacks with strategic effects should express human intent and must be subject to human accountability—does not require Washington and Beijing to share other principles of cybersecurity or infrastructure protection,” the researchers wrote.

The proposed agreement would require both countries to reduce the risk that chance, human error, unpredictable AI, or unauthorized actors could trigger attacks with strategic consequences.

The call for nuclear-style safeguards is part of a growing debate over how to slow AI development and keep increasingly capable systems under human control. The concern is especially acute in the military, where automated decisions made in milliseconds could turn a false warning or cyberattack into a nuclear crisis.

That concern was underscored in February when models from OpenAI, Google, and Anthropic used nuclear weapons in 95% of tested war-game scenarios, though experts cautioned that the simulation may have encouraged escalation.

That same month, Anthropic refused Pentagon demands to remove Claude’s restrictions on mass surveillance and fully autonomous weapons. Trump ordered federal agencies to stop using the company’s technology, but a federal judge ruled on Aug. 28 that the administration had illegally retaliated against Anthropic.

Calls for restraint have since spread across the industry. Earlier this month, Anthropic CEO Dario Amodei urged developers to slow capability gains, winning support from OpenAI CEO Sam Altman and xAI owner Elon Musk, while OpenAI asked Congress whether companies could coordinate a slowdown without violating antitrust law.

With no binding slowdown in place, the Brookings authors propose a narrower rule: Only humans should be allowed to authorize attacks on nuclear command systems or the use of nuclear weapons.

“The governance of military AI is not a negotiation that can afford to wait—it’s a race against time,” they wrote.