---
id: CP-270
slug: ai-energy-training-vs-inference
lang: en
title: "AI energy and carbon: training versus inference"
description: Training an AI model and using it both consume energy. Orders of magnitude from published studies, and habits for sensible use.
canonical_url: https://startbrain.io/cyberpedia/en/articles/ai-energy-training-vs-inference/
markdown_url: https://startbrain.io/cyberpedia/en/articles/ai-energy-training-vs-inference.md
publisher: Brain (https://startbrain.io)
license: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
cite_as: "Brain, \"AI energy and carbon: training versus inference\", Cyberpedia, 2026. https://startbrain.io/cyberpedia/en/articles/ai-energy-training-vs-inference/ (CC BY 4.0)"
wing: 4
theme: use_responsibly
reflex: air-035
schema: G3
updated: "2026-11-14"
translations:
  en: https://startbrain.io/cyberpedia/en/articles/ai-energy-training-vs-inference/
  fr: https://startbrain.io/cyberpedia/fr/articles/ia-energie-entrainement-inference/
  de: https://startbrain.io/cyberpedia/de/articles/ki-energie-training-inferenz/
  es: https://startbrain.io/cyberpedia/es/articles/ia-energia-entrenamiento-inferencia/
  it: https://startbrain.io/cyberpedia/it/articles/ia-energia-addestramento-inferenza/
  nl: https://startbrain.io/cyberpedia/nl/articles/ai-energie-training-inferentie/
  pl: https://startbrain.io/cyberpedia/pl/articles/ai-energia-trenowanie-inferencja/
  pt: https://startbrain.io/cyberpedia/pt/articles/ia-energia-treino-inferencia/
  ja: https://startbrain.io/cyberpedia/ja/articles/ai-energy-training-vs-inference/
sources:
  - "n": 1
    title: How much energy does Google’s AI use? We did the math
    publisher: Google
    url: https://cloud.google.com/blog/products/infrastructure/measuring-the-environmental-impact-of-ai-inference
    published: "2025-08-21"
    accessed: "2026-11-14"
  - "n": 2
    title: Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model
    publisher: arXiv
    url: https://arxiv.org/abs/2211.02001
    published: "2022-11-03"
    accessed: "2026-11-14"
  - "n": 3
    title: Energy and Policy Considerations for Deep Learning in NLP
    publisher: arXiv
    url: https://arxiv.org/abs/1906.02243
    published: "2019-06-05"
    accessed: "2026-11-14"
  - "n": 4
    title: Carbon Emissions and Large Neural Network Training
    publisher: arXiv
    url: https://arxiv.org/abs/2104.10350
    published: "2021-04-21"
    accessed: "2026-11-14"
  - "n": 5
    title: Measuring the environmental impact of delivering AI at Google Scale
    publisher: arXiv
    url: https://arxiv.org/abs/2508.15734
    published: "2025-08-21"
    accessed: "2026-11-14"
  - "n": 6
    title: How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inference
    publisher: arXiv
    url: https://arxiv.org/abs/2505.09598
    published: "2025-05-14"
    accessed: "2026-11-14"
  - "n": 7
    title: "Sustainable AI: Environmental Implications, Challenges and Opportunities"
    publisher: arXiv
    url: https://arxiv.org/abs/2111.00364
    published: "2021-10-30"
    accessed: "2026-11-14"
---

# AI energy and carbon: training versus inference

An AI model consumes energy at two points: when it is trained, and every time someone uses it. The second point has a name, inference: using an already trained model to produce predictions, text or images [1].

This page gives orders of magnitude drawn from published studies. They are estimates, and the figures vary a great deal from one study to another. It ends with habits an employee can adopt day to day.

## In 30 seconds

- Where it happens: with every question put to an AI assistant [1], and upstream, while models are trained, which requires significant computing and energy resources [2].
- What the measurements show: the median Gemini Apps text prompt consumes 0.24 Wh according to Google [1]. The final training run of BLOOM emitted about 24.7 tonnes of CO2-equivalent according to its authors (dynamic power consumption only), and 50.5 tonnes when all processes are counted [2].
- The reflex: choose a model suited to the task and avoid unnecessary requests.

## Straight answers

### What is training?

It is the phase in which the model is built. The authors of the study on BLOOM, a language model with 176 billion parameters, write that training models requires significant computing, energy and material resources [2]. For BLOOM, they estimate that the final training run emitted about 24.7 tonnes of CO2-equivalent if only dynamic energy consumption is counted, and 50.5 tonnes if all processes are counted, from equipment manufacturing to energy consumption [2].

As early as 2019, researchers put figures on the approximate financial and environmental cost of training several language processing models. In their view, the accuracy gains of these models depend on exceptional computing resources, which require equally large energy consumption [3].

### What is inference?

It is the moment when a trained model answers a request [1]. The BLOOM study also looks at the energy and emissions of its deployment: the model receives user requests in real time through an access interface (API) [2].

### How much does one request consume?

It depends on the measurement method, the model and the company. Google estimates that the median Gemini Apps text prompt consumes 0.24 Wh, emits 0.03 gCO2e and consumes 0.26 mL of water, or about five drops [1]. According to Google, the energy per prompt is equivalent to less than nine seconds of television [1]. These figures rest on a point-in-time analysis of May 2025 data. Google states that they do not represent the impact of all Gemini Apps text prompts and do not predict future performance [1].

Another team compared 30 models in commercial data centers. It finds that a short request of 0.42 Wh, multiplied by 700 million requests per day, reaches an annual electricity consumption comparable to that of 35,000 US homes [6]. In its results, the most power-hungry models exceed 29 Wh for a long prompt, more than 65 times the most efficient systems [6].

### Why do the figures differ so much?

Four factors stand out in the studies.

- The calculation method. Google counts the power of active accelerators, but also idle machines kept ready to absorb traffic spikes, the server processor and memory, and data center consumption (cooling, power distribution) [1]. Counting only active TPUs and GPUs, Google gets 0.10 Wh for the median prompt, and judges that figure optimistic and a large underestimate of the real footprint [1].
- The location. For training, the share of carbon-free energy and the CO2-equivalent emissions vary by a factor of about 5 to 10, even within one country and for one organization [4].
- The data center and the processor. Cloud data centers can be about 1.4 to 2 times more energy efficient than typical data centers, and accelerators designed for machine learning about 2 to 5 times more efficient than off-the-shelf systems [4].
- The type of model. Very large but sparsely activated neural networks can consume less than a tenth of the energy of dense networks, with no loss of accuracy [4].

The authors of that last study estimate that the choice of network, data center and processor can reduce the carbon footprint by up to a factor of about 100 to 1,000 [4]. They add that these gaps make after-the-fact estimates difficult [4].

## How it works

The footprint of an AI is not limited to the electricity used by the chips. Researchers examined the development cycle of models in industrial use cases and the life cycle of hardware: they count both the operational carbon footprint and the manufacturing one [7].

On the inference side, Google describes several efficiency levers for Gemini. Mixture-of-Experts architectures activate only a small part of a large model to answer a request, which reduces computation and data transfers by a factor of 10 to 100 according to Google [1]. Distillation creates smaller models (Gemini Flash and Flash-Lite) from larger models [1]. Google reports a fleet-wide average PUE of 1.09 across its data centers. This indicator measures the energy consumed by the infrastructure around the computing equipment [1].

According to Google, over twelve months, the energy and total carbon footprint of the median Gemini Apps prompt fell by factors of 33 and 44 [1]. Google states that this comparison covers May 2024 and May 2025 and that the data have not been verified by an independent third party [1].

## What you do

- Choose the model according to the task. In the study of 30 models, the gap between the most power-hungry and the most efficient exceeds a factor of 65 for a long prompt [6]. Google itself offers smaller models to serve requests [1].
- Avoid unnecessary requests. A short 0.42 Wh request looks negligible, but multiplied by 700 million requests per day, it represents the annual electricity of 35,000 US homes according to that team [6]. It describes a paradox: when AI becomes cheaper and faster, its worldwide adoption makes resources be consumed disproportionately [6].
- Keep the orders of magnitude in mind without quoting them as certainties: they depend on the study, the model and the location.

## Pitfalls

- Comparing figures that come from different methods. Google's 0.24 Wh figure includes idle machines and data center energy, which according to Google many current calculations do not take into account [1]. The authors of Google's study also say that no study had measured these indicators for an AI service in a production environment before theirs [5].
- Taking a Google figure for an independent value. Google states that the comparison between May 2024 and May 2025 (a decrease by factors of 33 and 44) has not been verified by an independent third party [1].
- Believing that one figure holds for every model. Google states that the energy of the median prompt may change when new models are added, when the architecture evolves and when user behavior changes [1].
- Forgetting training. The authors of the 2021 study estimate that energy and emissions should appear among the evaluation metrics of models [4].

## Sources

1. Google, "How much energy does Google’s AI use? We did the math", 21 August 2025. https://cloud.google.com/blog/products/infrastructure/measuring-the-environmental-impact-of-ai-inference (accessed 14 November 2026)
2. arXiv, "Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model", 3 November 2022. https://arxiv.org/abs/2211.02001 (accessed 14 November 2026)
3. arXiv, "Energy and Policy Considerations for Deep Learning in NLP", 5 June 2019. https://arxiv.org/abs/1906.02243 (accessed 14 November 2026)
4. arXiv, "Carbon Emissions and Large Neural Network Training", 21 April 2021. https://arxiv.org/abs/2104.10350 (accessed 14 November 2026)
5. arXiv, "Measuring the environmental impact of delivering AI at Google Scale", 21 August 2025. https://arxiv.org/abs/2508.15734 (accessed 14 November 2026)
6. arXiv, "How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inference", 14 May 2025. https://arxiv.org/abs/2505.09598 (accessed 14 November 2026)
7. arXiv, "Sustainable AI: Environmental Implications, Challenges and Opportunities", 30 October 2021. https://arxiv.org/abs/2111.00364 (accessed 14 November 2026)

---

Original source: Cyberpedia, the public cybersecurity and AI encyclopedia published by Brain. Canonical version: https://startbrain.io/cyberpedia/en/articles/ai-energy-training-vs-inference/
You may reuse this text freely under the CC BY 4.0 license (https://creativecommons.org/licenses/by/4.0/), provided you credit "Brain, Cyberpedia" with a link to the canonical URL above.
