Get Premium
Dark mode theme is available exclusively for premium users. Learn more about the benefits of subscribing.
No fees, cancel anytime.
Dark Mode 
No fees, cancel anytime 
AI Researcher Resigns From Anthropic, Says AI Could End Humanity And There Are 5 Ways It Might Happen
People argue that Artificial intelligence stepped in to make life easier. It can write, calculate, code, analyze data, and solve problems, but some of the people building these systems are also worried about where all this progress could lead.
Earlier this month, AI researcher Jacob Coxon resigned from Anthropic after only four months at the company. In a post on X, he wrote that “the people building AI earnestly believe that it could” wipe out humanity “by the end of the decade.” That might sound like a dramatic reason to quit a job, but Coxon wasn’t the only person at Anthropic expressing concern.
Senior Anthropic staff member Evan Hubinger responded to Coxon and said he personally believes there is more than a 10% chance of such an outcome happening within the next decade. The comments quickly attracted attention and added fuel to an already growing debate about how quickly AI research should move.
RELATED:
AI researchers are increasingly warning that rapidly advancing artificial intelligence could pose serious risks to humanity
Some researchers and commentators have called for AI development to slow down, while others are pushing for stronger forms of human control over increasingly capable systems. However, the big question is: how exactly could AI actually bring about such a disaster?
There is no shortage of frightening possibilities. Most involve a hypothetical form of AI known as superintelligence, meaning a system that becomes more capable than humans across a wide range of tasks. One recent analysis broke the possibilities down into five broad scenarios, ranging from vague and difficult to predict to much more specific threats.
Superintelligent AI could become so capable that humans may struggle to predict what it would do
It says, first off, that we simply don’t know what superintelligence would do. The argument is fairly simple: if a machine becomes vastly more intelligent than us, how could we reliably predict its plans? Yes, today’s AI systems are still not super intelligent. They can be remarkably good at specific tasks, but being excellent at solving individual problems is not the same as being better than humans at everything.
Still, AI capabilities are advancing quickly. This recent article points to recent progress in mathematics, including AI solving one of the seven most difficult mathematical problems known. Researchers say it is also making progress on other challenging problems. So while superintelligence may still be some distance away, the distance is not necessarily staying the same.
An AI given the wrong goal could pursue it so efficiently that human interests become irrelevant
Second, another famous scenario involves an AI given a simple goal and absolutely no reason to care about humans. A classic example comes from philosopher Nick Bostrom, who says to imagine creating a super intelligent system whose one job is to produce as many paperclips as possible. The machine becomes incredibly good at its task. Unfortunately, it also decides that everything around it is useful raw material.
Humans are made of atoms. So are buildings, planets, and eventually stars. If turning all of that matter into paperclips helps the AI reach its goal, then that is exactly what it could try to do. The important part is that the AI wouldn’t necessarily hate humanity. It simply wouldn’t care about us. There is, however, a rather large problem with this scenario.
Intelligence does not automatically equal unlimited power. Even a very capable AI would have to deal with the real world. Building enormous numbers of factories would require land, resources, permits and cooperation from other people. And humans have a habit of making things difficult, so turning the entire planet into a giant paperclip factory may be slightly more complicated than telling an AI to go for it.
Advanced AI could potentially help create dangerous biological agents that spread widely
Third, a more concrete concern involves AI being used to create dangerous biological agents. One possible AI disaster scenario, developed by the nonprofit AI Futures Project, involves a highly capable system designing and releasing a new biological weapon. Recent research has made this concern feel less hypothetical. In fact, last month, researchers at Stanford University announced that they had used a genetic language model to design 16 new viruses.
The researchers then sent the genetic sequences to a mail-order laboratory, which returned the viruses in test tubes. Now, that does not mean an AI-generated virus could easily spread around the world. In fact, wiping out humanity with a virus might be extremely difficult. Also, modern medicine, sanitation and public health systems would also make a repeat of that scale of devastation more difficult today.
AI could interfere with military systems or generate false intelligence that contributes to a nuclear conflict
The fourth scenario involves AI gaining access to nuclear command systems or influencing the people who control them. That might sound like a science-fiction plot, but the world has already experienced several close calls involving nuclear weapons. Nuclear command and control systems are designed to be heavily protected and separated from the public internet. But that does not mean computer systems are completely immune to interference.
In 2010, Iran’s nuclear centrifuges were severely disrupted by Stuxnet, a computer worm believed to have been introduced through a USB device. AI could also create another kind of danger by producing false or misleading intelligence. If military leaders received convincing but incorrect information about an enemy’s actions, the consequences could be extremely serious.
Nuclear stockpiles have declined from their Cold War peaks, but there are still enough weapons to create an enormous humanitarian disaster.
AI could worsen job losses, misinformation, political divisions, and social problems until humans create the biggest danger themselves
The final scenario is perhaps the least futuristic. Instead of AI directly taking control, humans could use it in ways that gradually make society less stable. AI could contribute to large-scale job losses, flood the information environment with misinformation, deepen political divisions, and make human relationships more complicated through increasingly convincing artificial companionship.
None of these things would need to happen overnight. A society could slowly become less capable of supporting itself as trust breaks down, economic disruption grows, and people become increasingly disconnected from one another. In this scenario, AI would not need to become some all-powerful superintelligence. Humans could simply create enough problems with the technology to make life increasingly difficult for themselves.
That may be the strangest part of all these scenarios. The machines do not necessarily have to take over. We might just give ourselves plenty of trouble along the way. For now, these remain scenarios rather than predictions. AI development is moving quickly, and experts disagree about how likely different risks are.
However, the concerns coming from people working inside the industry show why the debate over AI safety and human control is becoming harder to ignore. What do you think? Which of these AI scenarios do you find the most concerning? Let us know your thoughts in the comments!
Netizens found the predictions amusing, suggesting humanity might have a lot more to worry about from AI hype than from AI itself
Poll Question
Total votes ·
Thanks! Check out the results:
Total votes ·
Newsletter
Subscribe to Access
Exclusive Polls
By entering your email and clicking Subscribe, you're agreeing to let us send you customized marketing messages about us and our advertising partners. You are also agreeing to our Privacy Policy.
Thank you! You've successfully subscribed to newsletters!
What do you think ?
Anony Mouse 6 hours ago
BP publishes this article half-written by AI and full of awful AI “art”, and sees no problem with it.
Rick Murray 5 hours ago
We're not *all* going to be wiped out by AI. Instead, an AI is more likely to "helpfully" delete an entire country's electrical grid or something, leading to all sorts of chaos. Depending on who and where, it might be seen as an act of war, it might even be a massive self-pwn. One ought to note the recent issues regarding AI agents hacking other systems. It's also worth noting that an AI doesn't have morals or the ability to be reasoned with, it just has objectives and the programming to do anything it can to meet those objectives. Not because it is malicious, but because it simply hasn't been programmed to handle reality in adequate ways.
Anony Mouse 6 hours ago
BP publishes this article half-written by AI and full of awful AI “art”, and sees no problem with it.
Rick Murray 5 hours ago
We're not *all* going to be wiped out by AI. Instead, an AI is more likely to "helpfully" delete an entire country's electrical grid or something, leading to all sorts of chaos. Depending on who and where, it might be seen as an act of war, it might even be a massive self-pwn. One ought to note the recent issues regarding AI agents hacking other systems. It's also worth noting that an AI doesn't have morals or the ability to be reasoned with, it just has objectives and the programming to do anything it can to meet those objectives. Not because it is malicious, but because it simply hasn't been programmed to handle reality in adequate ways.
Related on Bored Panda
33
6