Why does AI use so much energy?

AI uses substantial energy because producing an answer requires many numerical operations on specialised chips, repeatedly moving model data through memory, then supporting that hardware with networking, power conversion and cooling. Large models, long prompts and long outputs generally require more work. Training adds a separate, concentrated energy cost; serving millions of later requests adds a continuing one.

Not every AI task is large. A small text model answering a short question can use a fraction of a watt-hour under recent estimates. The concern is the combination of demanding models, growing use and facilities built at unusual power density. The International Energy Agency says accelerated servers, mainly driven by AI adoption, are a major source of projected data-centre electricity growth in its Energy and AI analysis.

An answer is a chain of calculations

Language models store learned numerical relationships in parameters. When a prompt arrives, processors combine the input with those parameters through large matrix operations. The model then predicts a token, adds it to the context and repeats the process for the next token.

That token-by-token loop is why output length matters. Asking for a concise answer and asking for a book are not the same job. Input matters too: a long conversation or uploaded document must be processed before and during generation. Some reasoning systems also produce internal tokens that the user never sees.

The exact electricity depends on more than parameter count. Model architecture, numerical precision, memory access, software kernels and the accelerator generation all affect how much useful work is delivered per watt.

The chip does not simply “know” the model without physical work. Parameters and intermediate values have to move between high-bandwidth memory and processors. Memory capacity and bandwidth can become constraints, especially for large models. Keeping hardware fed with data costs energy even when the arithmetic itself is highly optimised.

This helps explain why a smaller or compressed model can be efficient. If fewer weights need to be read and fewer operations performed for each token, the workload can fall. Mixture-of-experts models can also activate only a portion of their total parameters for a token, although routing and serving design still matter.

The practical question is covered in do small AI models use less energy?. Size is an important signal, but it is not a complete power measurement.

AI hardware is power-dense

Modern accelerators are designed to perform enormous numbers of operations in parallel. A rack filled with them can draw far more power than a rack of conventional servers. Training and serving also connect many chips through fast networks, which add their own electricity demand.

The IEA notes that AI-focused data centres can have very large, geographically concentrated loads. This creates a different planning problem from the same amount of consumption scattered across a country. Grid connections, substations and generation have to be available at the right location and time.

EPRI’s Powering Intelligence scenarios reflect the uncertainty: future US data-centre demand changes widely depending on development and usage assumptions. Electricity forecasts are therefore best read as conditional paths, not promises.

The building around the chips also uses power

Servers are only part of a data centre. Cooling removes heat. Pumps and fans move air or liquid. Power systems convert electricity and provide backup. Storage, conventional processors and network equipment coordinate the workload.

Analysts often summarise this overhead with power usage effectiveness, or PUE: total facility electricity divided by electricity used by IT equipment. A PUE of 1 would mean no overhead, which real facilities cannot achieve. A lower PUE is better, but it does not say how efficient the AI model itself is.

This boundary matters when comparing the energy cost of one AI message. A chip-only estimate and a facility-level estimate can differ even if both calculations are internally sound.

Training is large; inference is relentless

Training exposes a model to huge datasets and updates its parameters across many processors. It can run for days or longer, with experiments and discarded training runs adding work beyond the final successful run. Public totals are rare, and comparing headline training figures is difficult when hardware, location and accounting boundaries differ.

Inference is smaller per task but happens every time somebody asks. A heavily used model can eventually consume more through inference than training; a lightly used model may not. Features that make multiple calls—search, agents, image analysis or repeated self-checking—can turn one visible action into several backend jobs.

This is why “one prompt” is not a durable technical unit. What the service does after the prompt is at least as important as the words typed before it.

More efficient AI can still mean more electricity

AI efficiency is improving through better chips, quantisation, sparsity, batching and model design. Luccioni, Jernite and Strubell’s comparison of inference workloads also shows that task and model choice matter: general-purpose generative systems can require substantially more energy than task-specific models for the evaluated tasks.

Yet a cheaper answer can lead to more answers. AI is being added to search, office software, coding, customer support and media generation. Total demand rises if the number of tasks grows faster than energy per task falls. That is how improved efficiency and rising electricity use can both be true.

The IEA’s base case projects global data-centre electricity consumption rising from about 415 TWh in 2024 to about 945 TWh in 2030. It attributes the growth to AI as well as other digital services, so the total should not be relabelled as AI alone.

What users and services can change

At the service level, the main levers are choosing the smallest capable model, improving utilisation, limiting unnecessary tool calls, using efficient hardware and locating flexible work where lower-carbon electricity is available. At the user level, concise prompts, sensible output limits and using ordinary software when it does the job can avoid needless computation.

Treechat routes everyday questions to a smaller model and estimates the energy of each response from model and token use. The receipt includes a stated allowance for data-centre overhead; it remains an estimate because providers do not expose a meter for each answer. Our methodology lists the current assumptions.

AI uses energy for understandable physical reasons, not because prompts are mysteriously converted into pollution. The useful response is to make the computational job visible, reduce it where possible and reserve bigger systems for work that genuinely needs them.