AI isn’t getting smarter, it’s getting more power hungry – and expensive

Follow ZDNET: Add us as a preferred source on Google.

ZDNET's key takeaways

• MIT estimated the computing power for 809 large language models.
• Total compute affected AI accuracy more than any algorithmic tricks.
• Computing power will continue to dominate AI development.

It's well known that artificial intelligence models such as GPT-5.2 improve their performance on benchmark scores as more compute is added. It's a phenomenon known as "scaling laws," the AI rule of thumb that says accuracy improves in proportion to computing power.

But, how much effect does computing power have relative to other things that OpenAI, Google, and others bring -- such as better algorithms or different data?

To find the answer, researchers Matthias Mertens and colleagues of the Massachusetts Institute of Technology examined data for 809 large language model AI programs. They estimated how much of each benchmark's performance was attributable to the amount of computing power used to train the models. 

They then compared that figure to the amount likely attributable to a company's unique engineering or algorithmic innovation, what they call the "secret sauce," which is sometimes -- but not always -- disclosed. And they compared general improvements in AI across the entire developer community and shared tips and tricks that consistently improve model performance.  

Their results are reported in the paper "Is there a 'Secret Sauce' in large language model development?", which was posted on the arXiv preprint server.

As Mertens and team framed the question, "Is the frontier of AI advancement propelled by scale -- ever-larger models trained on more compute? Or is it fueled by technological progress in the form of openly disseminated algorithmic innovations that raise performance across the field? 

"Or, do leading firms possess a genuine 'secret sauce' -- proprietary techniques that yield sustained advantages beyond scale and shared algorithmic progress?"

A lot more computing makes the biggest difference 

Spoiler alert: There is, indeed, a secret sauce, but it matters a lot less than simply having a bigger computer.

Mertens and team found evidence of all four helpful advances: more computing, secret sauce, general industry advances, and specific improvements of a given family of large language models (LLMs).

But the biggest difference by far was how much computing power was brought to bear by OpenAI and others. 

Also: AI killed the cloud-first strategy: Why hybrid computing is the only way forward now

"Advances at the frontier of LLMs are driven primarily by increases in training compute, with only modest contributions from shared algorithmic progress or developer-specific technologies," Mertens and team report. 

That means the best models will continue to result from scaling effects in compute, they conclude.

"As a result, sustained leadership in frontier AI capabilities appears unlikely without continued access to rapidly expanding compute resources.

"This implies that access to compute is central for AI leadership and helps explain the ongoing race to invest in compute infrastructure."

Specifically, a 10-fold increase in computing power has a measurable effect on a model's benchmark test accuracy, they found. 

"Models at the 95th percentile use 1,321× more compute than those at the 5th percentile," they relate, meaning that there's over a thousand times more compute used for the models that are better than 95% of models at benchmarks as there is for models at the lowest end of performance. That's a huge computing gap.

An important caveat is that Mertens and team were comparing open-source models, such as DeepSeek AI's, which they can examine in detail, with proprietary models, such as OpenAI's GPT-5.2, which is closed source and a lot harder to assess.

They relied on third-party estimates to fill in the blanks for proprietary models such as GPT and Google's Gemini, all of which are discussed and cited in a "Methods" section of the paper at the end.

(Disclosure: Ziff Davis, ZDNET's parent company, filed an April 2025 lawsuit against OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.)

Costs are going higher 

The study doesn't specifically identify the dollar cost of compute, but you can infer cost is going higher and higher. 

We know from other industry research that the cost of computer chips and related networking components required to scale up AI is generally on the rise. 

A study this week by the Wall Street brokerage firm Bernstein Research found that revenue for chip makers in 2025, including Nvidia, the dominant maker of GPUs powering AI development, reflected dramatic price increases across the board. 

After a slump in chip sales following the COVID-19 pandemic, the industry's sales finally returned to 2019 levels, wrote Bernstein chip analyst Stacy Rasgon, citing data from the industry's leading data provider, the World Semiconductor Trade Statistics.

Also: OpenAI's Frontier looks like another AI agent tool - but it's really an enterprise power play

But average chip prices in 2025 were 70% higher than in 2019, prompting Rasgon to observe, "Revenue growth over the last several years remains dominated by pricing." Chips are simply getting a lot more expensive, including the premium, he noted, for Nvidia's GPUs, and double-digit price increases for the DRAM memory chips from Micron Technology and Samsung on which LLMs depend, as I've noted previously. 

Simply put, it takes more money to make the next big computer for each new frontier AI model because it takes new chips that keep rising in price. Even if each new Nvidia Blackwell or Rubin GPU is more efficient than the last, which Nvidia frequently emphasizes, companies still have to buy enough of them to increase the total computing power at their disposal when developing the next frontier model.

That explains the hundreds of billions of dollars in capital investment that Alphabet's Google, Meta Platforms, and Microsoft and others are spending annually. It also explains why OpenAI CEO Sam Altman is in the process of raising tens of billions in financing and planning to spend over a trillion dollars.

Smart software can still lower costs 

The good news out of the study is that cost doesn't completely dominate, and engineering can still make a difference.

Even as the amount of compute dominates the frontier LLMs, technical progress in the form of smarter algorithms -- software, in other words -- can help reduce cost over time. 

The authors found that the smaller model developers, who have lower computing budgets generally, are able to use smart software to catch up to the frontier models on performance of inference, the making of actual predictions for a deployed AI model.

Also: How DeepSeek's new way to train advanced AI models could disrupt everything - again

"The largest effects of technical progress arise below the frontier," wrote Mertens and team. "Over the sample period, the compute required to reach modest capability thresholds declined by factors of up to 8,000x, reflecting a combination of shared algorithmic advances, developer-specific technologies, and model-specific innovations. 

"Thus, the secret sauce of LLM development is less about sustaining a large performance lead at the very top and more about compressing capabilities into smaller, cheaper models."

You could say, then, that for smaller firms, things are getting smarter in AI, in the sense that they use less power to achieve comparable results. Doing more with less is one valid way to define "smart" in the context of computing.

A world of haves and have-nots

All that confirms that it's a bifurcated world of AI, at the moment. To achieve greater and greater intelligence, one has to build bigger and bigger computers for ever-larger frontier models. 

But to deploy AI into production, it's possible to work on smaller models with better software and make them more capable within a limited computing budget.

Any way you slice it, giants such as Google, Anthropic, and OpenAI are likely to maintain their lead in the headlines of the most capable models at any point in time, thanks to their deep pockets.

Artificial Intelligence