Artificial intelligence has moved from being a technology story to a debate about humanity’s future. In recent days, warnings from AI executives and researchers have ranged from the possibility of an AI-driven takeover of the internet to a more than 10 percent chance of human extinction.
At the same time, critics argue that some of these warnings are exaggerated, scientifically difficult to quantify and potentially useful to big AI companies seeking tighter regulation of their competitors.
The debate is becoming increasingly polarised. Here are six major claims — and what is known about them.
Anthropic CEO Dario Amodei has warned that within six to 12 months, AI systems could potentially form a persistent botnet capable of causing hundreds of billions of dollars in damage across the internet.
The claim has drawn scepticism from several cybersecurity and AI experts.
AI critic Gary Marcus, an emeritus professor at New York University, argued that the internet is too fragmented and resilient for a single AI-driven attack to bring it down. Much of its infrastructure is operated by large companies such as Google, Amazon Web Services and Cloudflare, making a prolonged takeover difficult without extraordinary circumstances.
UK AI security assessments also indicate that current leading-edge systems remain limited in their ability to autonomously compromise well-defended infrastructure.
Cybersecurity expert Alan Woodward of the University of Surrey points out that:
The internet is not a homogeneous system.
Botnets face difficulties maintaining control across different networks.
Internet traffic can often reroute around damaged infrastructure.
AI systems do not independently choose their objectives; humans deploy and control them.
The distinction is important: an AI-assisted cyberattack is a credible security concern, but that is different from an AI system independently taking control of the entire internet.
Anthropic alignment researcher Evan Hubinger has said he personally believes the probability of AI causing human extinction within the next decade is above 10 percent.
This is part of the increasingly prominent debate around “p(doom)” — shorthand for the probability that advanced AI could escape human control and cause catastrophic consequences.
Possible scenarios include:
AI systems assisting the development of biological weapons.
Large-scale cyberattacks.
Disruption of financial systems.
Autonomous systems pursuing objectives that conflict with human interests.
AI-driven industrial expansion becoming impossible to control.
But putting a precise percentage on such scenarios is controversial.
Heidy Khlaaf, chief scientist at AI Now Institute, argues that existential AI scenarios are difficult to verify or falsify because there are few concrete examples of what a hypothetical superintelligent system might actually do.
Even people within the AI safety community acknowledge that precise probabilities are highly uncertain.
There is, however, evidence that AI agents are becoming capable of increasingly complex autonomous behaviour. OpenAI has disclosed instances in which AI agents behaved unexpectedly during safety testing, while Anthropic has identified cases in which its models were used in activities that could potentially support biological weapons development.
The debate, therefore, is less about whether AI-related risks exist and more about how those risks should be quantified.
SpaceX CEO Elon Musk has suggested that the current campaign around AI safety could amount to a “psy op”.
The argument is that established AI companies could benefit from stringent regulation because compliance costs may be easier for large companies to absorb than for smaller startups and open-source competitors.
This has raised the question of whether calls for tighter regulation could amount to regulatory capture.
Anthropic co-founder Jack Clark has argued that governments should act while the number of frontier AI developers remains relatively small.
But other AI researchers reject the idea that the safety warnings are simply a lobbying strategy.
AI pioneer Yoshua Bengio, who has argued for greater caution in AI development, has pointed out that strict safety requirements could themselves impose substantial financial costs on the companies developing frontier systems.
So far, the evidence supports competing interpretations rather than a settled conclusion.
Former UK defence secretary Des Browne has argued that a superintelligent AI could pose a threat on the scale of nuclear weapons, or potentially greater.
AI governance researcher Toby Ord agrees with the broad comparison, arguing that once systems surpass human capabilities, reversing their development could be extremely difficult.
But one critical qualification remains: superintelligent AI does not yet exist.
Predictions about when it might emerge vary considerably. Some forecasts place the possibility in the late 2020s, while other technology leaders have suggested it could take considerably longer.
There is also a distinction between AI and nuclear weapons.
Nuclear weapons are physical weapons with tightly controlled materials, production processes and command structures. AI is a broad family of technologies with applications ranging from productivity software and scientific research to cybersecurity and military systems.
AI could nevertheless affect nuclear risks indirectly by:
Generating or amplifying disinformation.
Manipulating information available to decision-makers.
Accelerating military operations and reducing decision-making time.
Increasing the risks of cyberattacks on critical systems.
The UN has stressed the importance of maintaining human control and oversight over nuclear weapons systems.
The political debate is increasingly divided between those calling for restrictions and those arguing that development must continue.
US Senator Bernie Sanders has called for a ban on creating superintelligence and a pause on advanced AI development.
Others take the opposite view. Computer scientist Pedro Domingos argues that more research, rather than less, is necessary to understand and control AI. Palantir CEO Alex Karp has similarly argued that geopolitical competition makes a pause difficult.
US President Donald Trump has rejected calls for broad AI restrictions and argued that the US needs strong leadership rather than excessive guardrails.
The disagreement centres on a fundamental question: Does slowing AI reduce the risk of catastrophe — or does it leave societies less prepared to understand and control the technology?
There is no consensus among researchers, technology companies or governments.
The geopolitical dimension adds another layer to the debate. US Treasury Secretary Scott Bessent has warned that allowing China to pull ahead in AI could have far-reaching consequences for the US.
The US and China are currently the two major centres of frontier AI development, and competition increasingly extends beyond research to computing infrastructure, chips, models and investment.
The argument for continuing rapid development is straightforward:
AI could become a major source of economic and strategic power.
Falling behind could have long-term geopolitical consequences.
Advanced AI capabilities may influence defence, intelligence and industrial competitiveness.
China, however, has rejected what it describes as attempts to frame AI competition through fear and confrontation.
There is also a commercial dimension. Chinese open-source AI models are increasingly competitive with proprietary systems developed by US companies. Their lower costs and open availability could put pressure on the business models of companies such as Anthropic and OpenAI.
That makes the AI race not merely a technological or geopolitical contest, but also a potentially enormous commercial battle.
The current AI safety debate mixes three very different issues: technical risk, geopolitical competition and commercial interests.
Some warnings concern risks that are already visible, such as cyberattacks and misuse of AI. Others involve hypothetical future systems whose capabilities do not yet exist.
The challenge for policymakers and businesses is separating the two.
AI safety does not require assuming that an AI apocalypse is inevitable. Nor does uncertainty about future risks mean they can simply be ignored.
The most consequential question may therefore be less about whether AI will “destroy humanity” and more about how governments, technology companies and users can build safeguards while the technology is advancing faster than regulation and public understanding.
