Sustainability in AI

This data was originally featured in the May 7th, 2025 newsletter found here: INBOX INSIGHTS, May 7, 2025: AI Integration Strategy Part 5, Sustainability in AI

In this week’s Data Diaries, let’s talk about sustainability. One of the questions that keeps coming up over and over again at events like the two-day workshop I just did for SMPS AEC.AI is how much of a sustainability impact AI has.

The reality is, we don’t know. Companies that have massive data centers don’t publicize just how much energy they use. But we do know how many GPUs have been sold, as a proxy for how much energy AI could be consuming. NVIDIA holds something like a 98% market share of GPUs in data centers, so if you go by the public estimates of how many data center GPUs NVIDIA has sold per year for the last 4 years, it looks like this:

• 2024: ~7.52M (based on earnings calls)
• 2023: 3.76M
• 2022: 2.64M
• 2021: 1.58M

All of these are modern datacenter GPUs, A100s through the current GB200 GPUs.

That’s 15.5 million GPUs. Export restrictions, especially to China (China and Taiwan make up about half of NVIDIA’s sales), started in 2022 and ramped up over the years. So call it half of those GPUs are likely in US data centers. Let’s make it 7 million for an even number, a little less than half.

Every NVIDIA “GPU” is actually a 8 core blade. If you look at the product specs, they’ve had 8 cores since the A100. That means with 7 million GPUs, you’re talking 56 million cores. Each core uses 700 watts. That’s JUST the core of the GPU. An 8 core GPU consumes 5,600 watts.

So just on cores alone, you’re at 39.2 billion watts. (7 million GPUs _ 8 cores each _ 700 watts per core)

But we don’t use GPU cores, we use GPUs. They all need cooling and they all have heat waste. For example, the DGX H100 pod that has 8 H100 cores in it has a peak usage of 10,200 watts, an overhead power consumption of 4,600 watts above and beyond the cores themselves.

So 7 million GPUs * 4,600 watts (because we accounted for the core power already) is another 32.2 billion watts.

So the total draw is 71.4 billion watts, SOLELY for the GPUs. This doesn’t count running the actual data centers, the HVAC, etc.

To put that in context, that’s 71,400 megawatts. The average USA home at any given time is consuming 20-30 kilowatts, which means that if AI chips in USA data centers are running full tilt, then AI is using the same amount of power as 2.86 million homes.

That begs the question, how do we reduce our AI power consumption? There are a few different ways to do this, to improve its sustainability.

1. Use the smallest model practical for any given task. AI companies typically provide a range of models – in ChatGPT, for example, you’ll see GPT-4o, o3, o4-mini, o4-mini-high. The smaller a model is, the less compute power it uses, so in OpenAI’s case, using GPT-4o is the lowest power consuming model. In Google Gemini, Gemini Flash or Gemini Flash Lite are the smallest models.

For tasks like summarization, extraction, rewriting – basically any task where you’re providing the data – a small model will get the job done just as well, but more efficiently in terms of energy use.

2. Run local models. Local models and local AI that run on your computer are even more efficient because, in comparison to big models that run in datacenters, small models run on laptops or even phones. Using free, open source software like AnythingLLM or LM Studio and models like Google Gemma 3 that you download and run, your power usage drops dramatically. Again, for those same core tasks where you’re providing most of the data, local models run just fine.
3. Do your heavy lifting off peak hours. Like all electrical usage such as air conditioning, consuming power at off-peak hours means utility companies and generators don’t need to spin up extra capacity. Some AI companies even give discounts; DeepSeek, for example, cuts its API fees by 50% if you use their services at off-peak hours. To the extent you can, schedule tasks that use a lot of compute for when providers aren’t as busy.

The bottom line for AI power usage is simple: use the smallest effective tool for the job. Just like you don’t need to fly a 747 to the grocery store, you don’t need to use the latest, greatest, biggest AI model for simpler tasks.

Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.