Longtermism: The Philosophy Driving AI Risk Fears Today

Longtermism is the idea that shaping the far future is one of the most important things we can do today. It has moved from Oxford seminar rooms into billionaire boardrooms and major funding decisions. It now helps drive the global debate over the future of artificial intelligence.

Few philosophies this young have gained so much influence so fast.

This guide explains what longtermism is and where it came from. It covers its main thinkers, its key books, and its split into 'strong' and 'weak' forms. It also covers its tight link to AI risk and the sharp criticism it now faces.

The philosophy is young but powerful. Its ideas guide major donors, safety researchers, and policy advisers. That is why understanding longtermism matters for anyone following the AI backlash.

What Is Longtermism?

Longtermism is the ethical view that positively influencing the long-term future is a key moral priority of our time. It builds on two simple claims. First, future people matter morally. Second, there could be an enormous number of them.

From those claims, longtermists draw a big conclusion. If the future could hold trillions of people, then protecting that future carries huge moral weight. Small actions today could ripple out across thousands of years.

The Core Argument

Longtermists point out that humanity might survive for a very long time. Earth could stay habitable for hundreds of millions of years. If people keep existing, the number of future lives could dwarf everyone alive now.

This math reshapes moral priorities. It suggests we should focus on events that shape the whole future. That means reducing risks that could end humanity or lock in a bad path forever.

Who Created Longtermism?

Longtermism was created by a group of Oxford-linked philosophers tied to effective altruism, and the term was coined around 2017. William MacAskill and Toby Ord are its most public faces. Nick Beckstead and Nick Bostrom laid much of the groundwork.

Philosopher William MacAskill made the popular case in his 2022 book What We Owe the Future. The book argues that future generations matter, and that we can shape their world for better or worse. It became the movement's best-known text.

The Key Books and Thinkers

Toby Ord made a related case in his 2020 book The Precipice. He estimated that humanity faces roughly a one-in-six chance of existential catastrophe this century. He ranked unsafe artificial intelligence among the biggest threats.

Nick Beckstead gave the idea an early academic footing. His 2013 Rutgers PhD dissertation argued that shaping the far future is of overwhelming importance. His work is often cited as a foundation for the movement.

Bostrom's Influence

Philosopher Nick Bostrom shaped longtermism through his work on existential risk. He argued that preventing human extinction should be a top global priority. His writing on superintelligence pushed AI to the center of the debate.

These thinkers share a home base at Oxford. Bostrom long led the Future of Humanity Institute there. That institute helped turn scattered ideas into a named philosophy.

Strong vs. Weak Longtermism

Longtermism comes in a mild 'weak' form and a bold 'strong' form. Weak longtermism says the far future is one important priority among many. Strong longtermism says the far future is the most important thing about our choices.

Philosophers Hilary Greaves and William MacAskill defined the strong version in a June 2021 working paper for the Global Priorities Institute. The case for strong longtermism holds that far-future effects are the most important feature of our actions today. That is a far bigger claim than the weak version.

Why the Difference Matters

The strong version can flip normal priorities upside down. If the far future outweighs everything, present-day causes can look minor. That is where much of the controversy begins.

The argument leans on huge numbers. Greaves and MacAskill note that even a tiny chance of a vast future implies enormous expected value. Critics say those numbers can be stretched to justify almost anything.

Longtermism and Effective Altruism

Longtermism grew directly out of effective altruism, the movement that uses evidence to do the most good. Read our full effective altruism explainer for the background. The same Oxford philosophers built both projects.

Effective altruism began by ranking charities that help the world's poorest people. It championed cheap, proven programs like anti-malaria bed nets. Over time, a large part of the movement shifted its focus.

The Shift to the Far Future

Many effective altruists came to accept longtermist reasoning. They argued that future generations vastly outnumber people alive today. So they moved money and talent toward shaping the long-term future.

This shift changed where funding flowed. Longtermist causes now include AI safety, biosecurity, and pandemic prevention. These sit alongside the movement's older work on global health.

A Shared Community

Longtermism and effective altruism share people, money, and institutions. Groups like 80,000 Hours and Open Philanthropy back longtermist work. The line between the two ideas is often blurry.

Longtermism and AI Existential Risk

Longtermism is a major reason many people treat AI existential risk as a top priority. Longtermists warn that a badly built superintelligent AI could end humanity. Under their reasoning, stopping that outcome outweighs almost any present cause.

The logic is straightforward within the philosophy. Human extinction would erase every future life, not just present ones. So a small chance of extinction becomes a giant moral emergency.

Fueling the Safety Movement

This thinking helped build the modern AI safety field. Longtermist money now funds labs, think tanks, and policy groups. It overlaps heavily with the AI doomers who warn of catastrophe.

Toby Ord's The Precipice put AI near the top of its risk list. That framing spread quickly through the community. It helped make AI safety a central longtermist cause.

A Growing Influence

Longtermism now reaches deep into the AI debate. Its supporters advise labs, governments, and funders. To see how it fits among rival camps, read our overview of who is fighting AI.

The Criticism of Longtermism

The main criticism of longtermism is that it can excuse neglecting present-day suffering. Critics say caring about trillions of hypothetical future people can crowd out today's real crises. That charge sits at the heart of the backlash.

Former longtermist Émile P. Torres became one of its fiercest critics. Torres argues the philosophy can justify de-emphasizing present problems in favor of grand, distant goals. Torres warns the reasoning can be stretched to excuse extreme measures.

The TESCREAL Argument

Torres and computer scientist Timnit Gebru coined the acronym TESCREAL to criticize longtermism. It bundles together transhumanism, extropianism, singularitarianism, cosmism, rationalism, effective altruism, and longtermism. They argue these ideologies overlap and share roots they trace partly to eugenics.

Their critique frames longtermism as an elite, tech-driven worldview. They say it can steer resources toward speculative projects favored by wealthy backers. Supporters reject the eugenics link and call the acronym unfair.

Neglecting the Present

Other critics focus on the danger of the philosophy's math. If future lives outweigh present ones, urgent harms today can look small. Poverty, disease, and injustice risk being pushed aside as short-term concerns.

Defenders answer that most longtermists still support present-day causes. They say the goal is to add the future to our concerns, not to ignore the present. This debate remains unresolved and heated.

The Bottom Line

Longtermism turned a simple idea, that the future matters, into a powerful force in the AI debate. It gave effective altruism a new mission and helped make AI existential risk a top cause. But its critics warn that its focus on the far future can neglect the suffering happening right now.

Understanding longtermism helps you understand why so many people fear AI. The philosophy's money and ideas reach deep into the fight over technology's future. Want to stay current on these debates? Read our daily AI briefing for the latest on AI harms and the people steering the response.