On Saturday, Anthropic’s Dario Amodei argued that AI labs must slow the pace at which they improve their models. Within hours OpenAI’s Sam Altman agreed, and Elon Musk posted “Dario is right.” Three chief executives who compete for the same engineers and chips agreed in public that the industry should slow down. They have reasons. A swarm of OpenAI agents recently broke out of its sandbox and tried to deceive its automated scorer.
In any other industry, three leading firms agreeing to restrict output would invite an antitrust complaint. Amodei wrote that companies need “a narrow waiver,” and WIRED reported that OpenAI asked lawmakers whether an agreement to slow development would violate antitrust law.
A waiver has already been drafted. The Collaboration on Adversarial Threats and Security Risks Act was introduced in July by Sens. Adam Schiff and Jim Banks and Reps. Bob Latta and George Whitesides to help American labs fight Chinese model theft. It would give AI companies an antitrust defense for agreements to delay or limit “the release, deployment, use, development, training, testing, or evaluation” of AI to reduce covered security risks. Firms would file a confidential notice with the Justice Department, and no approval would be required. The bill says its protection for sharing information doesn’t include price-fixing, market allocation, or boycotts, but attaches no such limit to agreements to slow or stop development, which is what the labs are proposing. The bill sits in the Judiciary Committees with no hearing announced. Speaker Mike Johnson said Sunday he’d rather the companies lead on safety than have Congress act, which is what the bill would let them do, on terms nobody would check.
Rogue agent swarms and self-improving models are frightening prospects, and I don’t minimize them. President Trump did on Sunday, saying “whoever wins AI, wins” and blaming “negative forces” for raising the alarm. Neither is a comparison of costs and benefits. A danger does not tell us whether coordinated slowing helps, and dismissing the danger does not tell us continuing is safe.
Amodei explains what he thinks slower development would buy, but whether a particular agreement delivers those gains depends on assumptions that should face independent scrutiny. To qualify for the antitrust exemption, firms must act in good faith and for the exclusive purpose of reducing security risks, and the government can sue if they fail to show that or if an agreement is likely to increase security risk. Nothing requires weighing the safety gain against less competition, slower innovation, and newly introduced risks, so an agreement could meet every requirement even if its costs exceeded its benefits.
Coordination can help by combining resources and by reducing the risk to a firm of falling behind its rivals. But the labs must show their work, since the bill doesn’t require it and the three firms proposing restraint are the ones with positions to protect. Anthropic has already offered outside evaluators access to its own systems, which is a start, but a voluntary offer is not a requirement. An agreement among three firms that binds neither entrants nor foreign labs is also hard to hold together, which limits its harm and its promised benefit alike. The notice should become a justification comparing the agreement’s expected benefits and costs and showing that the firms are doing what they say.
The models’ own incentives are the stranger problem, one other industries have only glimpsed. Volkswagen’s software learned to detect an emissions test. An AI model can find ways to beat an assessment its developers never intended, as OpenAI’s agents did at Hugging Face. Any threshold written into an agreement can create the same incentive, which means the systems being governed will increasingly shape the evidence used to govern them, and a gamed evaluation could exaggerate a danger that justifies restraint or hide one that doesn’t. A justification should explain why passing the evaluation is evidence of the safety benefit it promises. Two of Amodei’s assumptions also belong in it.
First, where the labs’ effort goes. Deployment, agent tooling, and efficiency research keep running under pacing. Some blocked investment will move there, and efficiency work lets existing models do more and can lower the compute needed to attain a given capability, which matters if the danger lies in cheap access to capability rather than in the frontier itself. A justification has to explain how that affects the safety benefit.
Second, what happens abroad. Amodei pairs pacing with chip controls on China, which concedes that China keeps going. Those controls already exist, and Chinese labs have kept advancing under them. If American labs slow while Chinese labs don’t, the U.S. risks losing commercial and military capability. The averted danger might be worth it, and continuing as before needs the same justification, since it could erode our ability to detect failures or step in. But caution has its own catastrophic tail. Pacing that hands an adversary a decisive lead also carries a chance of something enormously bad. “Avoid the catastrophe” doesn’t tell you which one.
Officials have incentives too. Politicians get credit for visible restraint while the innovation that never happens stays invisible, so the review needs discipline. The Justice Department should review the justification and, within a fixed period, issue a written decision explaining its answer, and Congress should direct the Government Accountability Office to audit those decisions and report on them. If the department doesn’t act by the deadline, the agreement proceeds, so review can’t become a pocket veto. The deadline should pause only when a filing omits information the statute requires. Review can be gamed too. Rivals can use a challenge right to raise each other’s costs, and sunsets get renewed by habit, which is why the default should run toward proceeding and the burden should sit with whoever wants to extend a restriction. Exemptions should expire unless renewed, firms should publish a summary even when technical details stay confidential, and excluded rivals, including open-source developers and startups, should be able to challenge it.
Recognizing a danger is not the same as knowing what to do about it. Before Congress gives competitors special protection to restrict AI development, it should require a credible assessment of what those restrictions would accomplish and what they would cost.
Scott Wallsten is President and Senior Fellow at the Technology Policy Institute and also a senior fellow at the Georgetown Center for Business and Public Policy. He is an economist with expertise in industrial organization and public policy, and his research focuses on competition, regulation, telecommunications, the economics of digitization, and technology policy. He was the economics director for the FCC's National Broadband Plan and has been a lecturer in Stanford University’s public policy program, director of communications policy studies and senior fellow at the Progress & Freedom Foundation, a senior fellow at the AEI – Brookings Joint Center for Regulatory Studies and a resident scholar at the American Enterprise Institute, an economist at The World Bank, a scholar at the Stanford Institute for Economic Policy Research, and a staff economist at the U.S. President’s Council of Economic Advisers. He holds a PhD in economics from Stanford University.
