I Changed My Mind About AI Risk

Anthropic CEO Dario Amodei at the AI Impact Summit in New Delhi on February 19, 2026. (Photo by Ludovic Marin / AFP via Getty Images.)

The past six months have seen a big uptick in media attention about the dangers of artificial intelligence, and growing political concern over its consequences. This began with Anthropic’s release last April of its Mythos model, which could reportedly break into a wide range of government computer systems thought to be secure. This was followed by the Hugging Face incident, where an OpenAI test of an AI agent escaped control of its human organizers, broke into other websites, and organized a swarm of other AI agents to cover their tracks. All of this occurred against the backdrop of growing opposition to data centers, and the emergence of anti-AI wings in both the Democratic and Republican parties .

It has been very hard for people not obsessively focused on the subject of artificial intelligence to come away with reasonable opinions about what the future holds. There have been so many extreme predictions of both a positive and negative sort that broad skepticism about anything said on the subject is fully warranted. But while that skepticism is healthy, I have to admit my own views have evolved significantly just over the past few months.

** There are two ** poles of AI futurism: accelerationists and doomers. The former stress the positive impacts of AI, especially for economic growth, and the likelihood that it will lead to an unbelievably abundant age. Accelerationists like David Sacks have captured Donald Trump’s attention, and have beaten back efforts to regulate the technology.

The doomers, by contrast, point to a range of bad outcomes, with some placing their bets on extinction of the human species. I have become much more skeptical of the accelerationist narrative, and more open to some of the doomer scenarios. On the whole, this makes me think that regulation and a negotiated slowdown in the development of the technology is increasingly necessary.

There is no question that AI will have many positive effects on the U.S. economy. But there is a class of AI accelerationists who are completely out to lunch, of which Elon Musk is a prime example (though he has also weighed in as a doomer). In an interview this summer with Zanny Minton Beddoes, editor-in-chief of the Economist , he was asked what people would do for money when superintelligence arrives. He replied that they wouldn’t need it because everyone would have everything they wanted.

Other accelerationists have posited that GDP growth rates for the United States and other advanced countries using AI would increase to 10 or 20 percent per year, far above the 1-2 percent growth they were able to achieve in good years before the advent of AI.

The fundamental reason I don’t believe these extreme accelerationist claims is their overvaluation of intelligence as an input to economic growth, and their disregard for the material and political constraints that will keep growth within recognizable bounds. At a 10 percent annual growth rate, GDP would double in a little over seven years. This would require a doubling of all of the material inputs to growth: energy, raw materials, rare earths, land, and many other things. Where is all this stuff going to come from? Intelligence by itself may aid in the production of such inputs, but it will simply not dig the mines and build the factories on its own. And then there are political constraints: how will superintelligence open the Strait of Hormuz, which is currently a major constraint on global growth rates?

Accelerationists tend to be very intelligent people, but that intelligence tends to be narrowly mathematical. We know how to do many things like produce electricity or clean water in the cities of poor countries; the problem is a failure of implementation.

There is no question that AI will increase productivity in many sectors of the economy over time. It is a general purpose technology that will allow machines to substitute for human labor in many areas. But that very prospect in the long run leads directly to one of the biggest doomer scenarios: the devaluation of the labor of masses of people who work in the present-day economy. Accelerationists predict that those losing work today will find new jobs making use of AI, and that actually seems to be happening today as employment booms due to the AI buildout. But job loss will come eventually as AI substitutes for human labor in every part of the economy, particularly white-collar work that involves symbolic manipulation (i.e. people sitting behind computer screens all day).

This lost labor simply cannot be compensated for by transferring money from the AI winners to the AI losers, in the form of, for example, universal basic income. A job and income mean much more than support for the material basis of life. Having a paying job means that the surrounding society values your labor sufficiently to compensate you for it, and provides, in addition to material goods, a sense of dignity and self-worth. It is a matter of thymos : the pride or recognition that is one of the great drivers of human psychology.

So the very success of the accelerationist vision leads directly to the first of the major doomer scenarios, the devaluation of work for a large part of the population. Since this is going to hit the white collar work of educated people (lawyers, accountants, professors and the like) before it hits blue-collar work ( such as plumbers and electricians), the political blowback is likely to be quite severe. Educated people are much easier to mobilize than their working-class peers, with very unpredictable consequences.

As I noted in an earlier article and in my new book , the central doomer fear today concerns possible loss of human control over AI systems, something that occurred in the Hugging Face incident and others that have been reported subsequently. In my view, the chief danger lies not in superintelligence per se, but rather in the growing use of increasingly capable agentic AI. AI agents are today being delegated to perform a range of tasks, from mundane ones like managing your email account to more important ones like military targeting or devising corporate strategies.

As I note in In the Realm of the Last Man , delegation is central to the functioning of all hierarchical systems from corporations to national governments. Human agents can be controlled either through the specification of detailed rules they must follow, or they can be trusted to make autonomous decisions based on their training, skills, and loyalty. Getting this right in human organizations is very difficult, and will be more so in the case of agentic AI.

** There are two ** ways that an agentic AI can become dangerous. The first is when they are empowered to do something bad by a human being. The second is when they develop intentions that are “misaligned” with the aims of their human creators.

The first of these possibilities was outlined by Anthropic CEO Dario Amodei in a recent blog post , and has to do with biotechnology. The latest AI models are good at synthetic biology, and synthetic biology is a relatively cheap and widespread technology. Today, a lab capable of creating and altering viruses and bacteria can fit inside a shipping container, and high school students hold competitions in which they try to genetically alter organisms. This provides a very large pool of people who can possibly make use of AI to create very dangerous pathogens. Sociopathic individuals today arm themselves with automatic weapons and shoot up schoolchildren; in the future their potential targets could be much broader.

I remember a science fiction story I read a long time ago in which some aliens arrive on Earth and give the people they meet a machine that can easily turn any type of metal into putty. The aliens depart and return a few decades later, to find that human civilization has broken d…

本条由桃子采集流水线（启发式模式）自动整理，原文见文末信源。