Is AI Overhyped? Big Claims Checked Against the Evidence

Is AI Overhyped? The Short Answer

Partly. Artificial intelligence (AI) tools are useful for some narrow tasks, but claims about replacing workers and transforming companies run ahead of the measured results. If you doubt the AI sales pitch, the table below sets each claim against the study or company figure that tests it.

Studies of single tasks find real gains. Studies of whole companies and whole labor markets find small effects.

None of these sources say AI is worthless. The sources measure specific jobs and specific outcomes, and each number applies only to what it measured.

Why Most Company AI Projects Stall

The 95% figure is not a measured failure rate for AI projects. The MIT NANDA report, "The GenAI Divide: State of AI in Business 2025," says "95% of organizations are getting zero return." The report sets that against $30 to 40 billion in enterprise investment in generative AI.

The figure rests on a small, preliminary sample. The authors used 52 structured interviews, a review of more than 300 public AI initiatives, and surveys with 153 leaders. The report is labeled "Preliminary Findings," and the authors call the figures "directionally accurate based on individual interviews rather than official company reporting."

The report also gives a funnel for enterprise-grade tools. Sixty percent of organizations evaluated such tools. Twenty percent reached pilot stage, and 5 percent reached production.

IBM's CEO survey points the same way. Only 25% of AI initiatives delivered expected ROI, and only 16% scaled enterprise wide.

The report also found that only 40% of companies said they bought an official large language model (LLM) subscription. Yet workers at over 90% of the surveyed companies reported regular use of personal AI tools for work.

Only 5% of enterprises have AI tools integrated in workflows at scale. The report says 7 of 9 sectors show no real structural change.

A mid-market manufacturing chief operating officer (COO) quoted in the report said online hype claims everything has changed. In that COO's operations, nothing fundamental had shifted.

The Register's August 2025 article gives the investment range as $35 to 40 billion. The report itself says $30 to 40 billion.

Do AI Tools Make Workers Faster?

The studies disagree, and they measure different things. Some measure how fast a task gets done. Others measure pay and hours across a whole labor market.

In METR's 2025 study, 16 experienced open-source developers worked on 246 real issues. With AI tools allowed, they took 19% longer. Before the study, they expected AI to speed them up by 24%.

Afterward, they still believed AI had sped them up by 20%. METR calls the result "a snapshot of early-2025 AI capabilities in one relevant setting." The study does not show that AI slows down most developers.

METR's February 2026 update used late-2025 tools. Returning developers showed an estimated 18% speedup, with a 95% confidence interval from a 9% slowdown to a 38% speedup. Newly recruited developers showed an estimated 4% speedup.

METR called this "only very weak evidence" of a speedup. It also said its study design is no longer reliable for measuring current effects. METR said selection bias likely makes the estimate a lower bound, because many developers would not work without AI.

A 2023 working paper from the National Bureau of Economic Research (NBER) found a gain on a different kind of work. Its authors were Erik Brynjolfsson, Danielle Li and Lindsey R. Raymond.

An AI assistant raised customer-support agents' productivity, measured as issues resolved per hour, by 14% on average. The gain reached 34% for novice and low-skilled workers. Experienced and highly skilled workers saw minimal impact.

Anders Humlum and Emilie Vestergaard studied 25,000 workers at 7,000 workplaces in 11 AI-exposed occupations in Denmark. They found AI chatbots had no significant impact on earnings or recorded hours in any occupation.

The authors estimated average time savings of 3%. They say that modest gain, combined with weak wage pass-through, helps explain the limited labor market effects.

Their paper also notes that controlled trials show productivity effects of 15 to 50 percent. It adds that effects can turn negative when AI is applied to the wrong tasks.

The jobs side of this debate has its own guide, on AI and jobs. For the developer studies specifically, see whether AI can replace developers.

Where AI Does Deliver

The measured gains in this set come from narrow tasks. Customer support is one example, and the controlled trials cited above point the same way.

Klarna's February 2024 press release describes a large customer-service result. The AI assistant handled 2.3 million conversations in its first month, about two-thirds of Klarna's customer-service chats.

Klarna said the assistant did the equivalent work of 700 full-time agents. It projected about $40 million in added profit for 2024, and that profit figure was a projection.

In May 2025, Klarna's CEO, Sebastian Siemiatkowski, said Klarna would hire human agents again. He said, "there will always be a human if you want."

Klarna's headcount fell 22%, to about 3,500, during a hiring freeze of more than 12 months, mostly through attrition. For the layoff side of that story, see our guide to AI layoffs and whether they backfired.

In the MIT NANDA report, over 80% of organizations explored or piloted general tools such as ChatGPT and Copilot, and nearly 40% report deployment. The MIT NANDA report says these tools primarily enhance individual productivity, rather than profit and loss (P&L) performance.

IBM's survey also found that 64% of CEOs acknowledge the risk of falling behind drives investment in some technologies before they have a clear understanding of the value.

The evidence supports AI for specific tasks. It does not show that those gains have added up to company-wide change yet.

The Hype Cycle

Gartner's 2025 Hype Cycle for AI, as reported by Today's General Counsel, says generative AI "has officially begun its descent into the Trough of Disillusionment."

The same coverage reports that, despite an average $1.9 million in generative AI investment during 2024, fewer than 30% of AI leaders say their CEOs are satisfied with the returns.

That placement comes from secondary coverage of Gartner's report. That coverage doesn't give a 2026 placement.

For the investment side of the same argument, see our guide to the AI bubble.

How Many People Use AI

A Pew Research Center survey of adults in the United States, fielded February 17 to 23, 2026, found 49% use AI chatbots. That is up from 33% in 2024. 51% do not use them.

Pew found that 24% use chatbots daily. It also found that 38% of employed adults use them for tasks at work.

Pew's 2026 survey also found that 63% say AI is advancing "too quickly." 40% expect its impact on society over the next 20 years to be negative, and 16% expect it to be positive.

Pew's October 2026 follow-up asked non-users which reasons were major reasons for not using chatbots. 60% said they are not interested in using chatbots, and 54% worry about how their personal information will be used.

Another 45% said they don't trust chatbots to give accurate information, and 29% said they don't know how to use them.

Census Bureau data on United States businesses put overall AI use at 17% to 20%. Between 20% and 23% of businesses expected to use it in the next six months.

The split is wide by firm size. 37% of firms with 250 or more employees used AI, and fewer than 20% of firms with four or fewer employees did.

In November 2025 the Census Bureau changed its question, from AI use "in producing goods or services" to AI use "in any business function." Earlier and later figures are not directly comparable.

Microsoft's May 2026 analysis estimates that 17.8% of the world's working-age population, ages 15 to 64, used a generative AI product in the first quarter of 2026. That is up from 16.3%. The United States figure was 31.3%.

The Microsoft estimate uses Microsoft telemetry, adjusted for device share, internet access and population.

The 84% figure comes from a viral chart. The chart is titled "AI users in Feb 2026," and it spread across social media. A repost by DeepakNess gives no method or sources, and the original author is unconfirmed.

The chart counts the whole world population, including children. Microsoft's 17.8% counts only working-age people, so the two figures use different bases.

Subtracting 17.8% from 100% gives about 82%, but Microsoft does not publish a non-user share.

How to Judge the Next AI Claim

Use these questions on the next AI claim. Each one comes from a gap in the evidence above.

1. Ask what the number measures: one task's speed, a team's output, or profit and loss (P&L). The MIT report says general-purpose tools primarily enhance individual productivity.
2. Ask how many people the figure rests on, and whether the authors call it preliminary. The MIT figure rests on 52 interviews and 153 surveyed leaders.
3. Ask which tools were tested, and when. METR's 2025 result came from early-2025 tools, and its 2026 follow-up used late-2025 tools.
4. Ask whether a figure is a projection or a result. Klarna's $40 million profit figure was a projection.
5. Ask whether a forecast has a date that can be tested. Amodei's "three to six months" window is one example, and Ryan Greenblatt says he does not think it came true.
6. Ask whether the number has a published method. The 84% chart gives no method, while Microsoft lists its adjustments for device share, internet access and population.
7. Ask whether the result is split by experience. Brynjolfsson and colleagues found a 34% gain for novices and minimal impact for experienced workers.

For the spending side of the argument, see our guide to the AI data center bubble.

Sources

• MIT NANDA, "The GenAI Divide: State of AI in Business 2025" (PDF)
• The Register, "Generative AI does nothing for 95 percent of companies" (August 18, 2025)
• METR, early-2025 developer productivity study (July 10, 2025)
• METR, uplift study update (February 24, 2026)
• Today's General Counsel, on Gartner's AI Hype Cycle (August 8, 2025)
• IBM Newsroom, CEO study (May 6, 2025)
• Klarna press release (February 27, 2024)
• Entrepreneur, on Klarna hiring human agents (May 9, 2025)
• NBER Working Paper 31161, "Generative AI at Work"
• Becker Friedman Institute Working Paper 2025-56, "Large Language Models, Small Labor Market Effects"
• Pew Research Center, "Americans and AI 2026" (June 17, 2026)
• Pew Research Center, "Why Don't Some People Use Chatbots?" (October 9, 2026)
• U.S. Census Bureau, AI use by businesses (May 26, 2026)
• Microsoft On the Issues, "The state of global AI diffusion in 2026" (May 7, 2026)
• DeepakNess, repost of the "AI users in Feb 2026" chart (March 7, 2026)
• Council on Foreign Relations, CEO Speaker Series with Dario Amodei (March 10, 2025)
• Redwood Research blog, "Is 90% of code at Anthropic being written by AIs?" (October 22, 2025)