AI and the extinction of humanity: the $2 trillion danger?

Science without conscience is but the ruin of the soul. Rabelais could not have been more right, five centuries before the advent of artificial intelligence. Sam Altman, Geoffrey Hinton, Barack Obama, the UN, and even Vitalik Buterin all express concern, on paper, about the same danger. That of an artificial intelligence that escapes all control. However, as soon as you scratch the surface, no one is talking about the same thing. One fears the extinction of the human species. Another worries about the disappearance of millions of jobs, the concentration of power in three American companies. Or simply that their Chinese neighbor gets too far ahead. The same word, alert, covers three logics that have almost nothing in common, except for the vocabulary they borrow.
Key points of this article:

• Several influential figures, including Sam Altman and Geoffrey Hinton, have expressed their concerns about the possibility of artificial intelligence escaping human control.
• A persistent debate opposes those who fear the extinction of the human species due to AI and those who are concerned about the immediate consequences on employment and algorithmic biases.

Extinction or destroyed jobs: artificial intelligence is not feared for the same reasons

In March 2023, the Center for AI Safety, a California research institute, had a unique phrase signed by over 350 leaders and researchers. It elevates the fight against the extinction risk related to AI to a global priority. On the same level as pandemics or nuclear war.

Among the signatories are Sam Altman, Demis Hassabis, and Dario Amodei. These three leaders head the most advanced laboratories in the sector, namely OpenAI, Google DeepMind, and Anthropic, respectively. Geoffrey Hinton and Yoshua Bengio, who laid the foundations of deep learning twenty years ago, also signed.

Two and a half years later, in October 2025, the Future of Life Institute pushes the cursor further with its Statement on Superintelligence. And once again, more than 800 signatures, including those of Steve Wozniak, Richard Branson, and Prince Harry. They demand a ban on the development of superintelligence until a scientific consensus and public acceptance guarantee its safety. Bengio is still involved.

Alongside these two texts, another group of researchers has been working for years on a completely different register. That of algorithmic biases in recruitment or predictive justice. It also concerns the precariousness of entire professions and the handful of American companies that concentrate global computing power. For this camp, often grouped under the label of AI ethics, talking about human extinction diverts attention from already measurable damages, those affecting specific workers today, not an abstract humanity fifty years from now. Both camps use the word risk, but they are not talking about the same risk.

Altman and Musk: the arsonist firefighter

Let’s return to our good signatories. Sam Altman signs the Center for AI Safety text in May 2023. That same year, OpenAI raises $10 billion from Microsoft and begins the race for the most powerful model on the market.

In 2026, the company files for an IPO without ever slowing down in the race for computing power. However, just as we are writing this article, it announces this very morning that it is pulling back for this year. Alerting about the extinction of the species and chasing after the next funding round forces this ambiguity.

Elon Musk is not to be left behind. In March 2023, he also co-signed the letter from the Future of Life Institute. It calls for a six-month pause on training the most powerful models. Four months later, xAI was born. Grok, its model, is now running on one of the largest computing clusters ever assembled. The pause has never been applied by anyone.

At Anthropic, Dario Amodei regularly publishes texts detailing, with supporting figures, the scenarios in which a frontier model could escape the control of its creators. At the same time, Anthropic aims for a record valuation for its IPO. The company has also won a highly publicized tug-of-war with the Pentagon over the military use of its own models.

One might almost be tempted to see it as a remake of the Manhattan Project. The physicists who built the bomb were also the first. Once Hiroshima and Nagasaki were wiped off the map, they called for international control of the weapon they had just delivered. The comparison has its limits. Oppenheimer never negotiated a share of royalties on plutonium. The three AI leaders alert while, every quarter, activating a capital-raising machine.

We Must Pace the Frontier: I've written a new essay on why the AI industry should slow down, with a three-part plan for doing so.

Anthropic is unilaterally committing to the first of these steps. We'll provide third-party evaluators with permanent, employee-level access to our...
--- Dario Amodei (@DarioAmodei) September 12, 2026
<< We must push the boundaries: I have written a new essay explaining why the AI industry should slow down, proposing a three-point plan to achieve this. Anthropic is unilaterally committing to implementing the first of these steps. We will grant third-party evaluators permanent access, equivalent to that of our employees, to our systems. They will be able to verify compliance with our safety measures, report incidents, and assess the consistency of models during their training. >>

Dario Amodei

Obama, Trump, Beijing: The Race for Artificial Intelligence Disguised as a Security Framework

In the face of this endless race, politics gets involved. Barack Obama urges American Democrats to have a clear plan on AI. He also calls, according to the New York Times, for a form of caution regarding the speed of model deployment. The statement comes as the Trump administration pushes in exactly the opposite direction. A presidential decree aims to prevent each U.S. state from legislating on AI independently, in favor of a single, more permissive federal framework. A previous decree from Trump had already promised to marry security and innovation. In practice, the balance clearly tilts towards the latter option.

On the Chinese side, the official discourse uses the same vocabulary as Obama or the Future of Life Institute. At the 2026 World AI Summit in Shanghai, Beijing signs a declaration in favor of coordinated global governance of the sector. At the same time, a Chinese company is accused of having plundered Anthropic's models to catch up, which led to a backlash from Washington. The displayed cooperation and the real technological race advance on two tracks that never cross.

The European Union has chosen the opposite path, enforcing the AI Act, which has been implemented in phases since 2024. Brussels regulates a market where it neither possesses the most powerful models nor the data centers. Washington and Beijing fund AI. Europe comments on it.

Mirror, mirror on the wall, who is the fairest AI?

In July 2026, the Federal Communications Commission (FCC) authorized Reflect Orbital. This American startup will test a giant mirror in low Earth orbit. It is capable of reflecting sunlight to a specific point on Earth at night. The stated goal is to sell sunlight on demand to farmers or construction sites that want to extend their working days. Astronomers, biologists, and several night sky advocacy groups have been sounding the alarm for months. They fear the light pollution that a large-scale deployment would cause. They also point to the disruption of wildlife sleep cycles and the potential rendering of observatories unusable. The CBC has documented the legal gray areas surrounding the issue. No international law truly governs who has the right to alter the night of a neighboring country from space. Nevertheless, the FCC approved the project.

This has nothing to do with a language model, at least on the surface. However, the same mechanism is at work on both sides. A scientific community raises alarms, documents, and quantifies the risk. A private entity, supported by a regulator that has the final say, moves forward regardless, because it has the technical capability and funding. The alert has never been the problem. The problem is that no one, neither the 800 signatories of the Future of Life Institute, nor the researchers at the Center for AI Safety, nor Barack Obama, has the power to stop those who decide to press the button.

Reflect Orbital and AI: The Same Power Mechanism

This is where the comparison with Reflect Orbital becomes more useful than any parallel with the atomic bomb. An orbiting mirror and a foundation model with several hundred billion parameters have, on paper, nothing in common. Yet they share the same decision-making structure. A handful of private actors, a regulator that authorizes them more often than it restrains them. This mass of alert texts accumulates without ever weighing on the deployment schedule. The FCC took a few months to decide on Reflect Orbital. The same texts have been circulating about AI for three years, with no laboratory having slowed its roadmap by even a notch.

The UN itself acknowledged in a report published this year that AI could cause catastrophic damage if no binding control mechanism emerges by the end of the decade. The text has no legal value. No state is obliged to implement it.

Reflect Orbital is set to launch its first test mirror before the end of the year. AI laboratories have already scheduled their next models for 2027. The alert texts will continue to accumulate on the desks of people who have never had the power to enforce them.