⚡ Power, Heat, and Intelligence ☁️ - AI Data Centers Explained 🏭

🔎 GENERAL INFO 🔎

1. How do data centers work? 🏭

Data centers are the beating heart of AI, enabling three things: compute, storage and networking, by housing and connecting all of the infrastructure needed to fulfill these functions, and ensuring that this infrastructure remains stable and secure. At a macro level, data centers are structured on multiple scales: from boxes (which host individual servers) to racks (which contain multiple servers), which are, in turn, housed in buildings that are grouped together into campuses, and then into regions (such as “us-east-1”).

To ensure their integration with the world at large, data centers are connected via routers and cables to other servers, allowing information to be transferred both locally and globally. And while most data centers now consist of football field-sized warehouses full of servers, there are also smaller configurations depending on the needs of the company that owns them.

2. What makes AI data centers unique? 🤖

With the advent of AI, the requirements of data centers have shifted - while much of the burden was previously placed on the storage and transfer of data, nowadays the emphasis is very much put on parallel, compute-intensive processes. Calculations such as matrix multiplications, required by modern-day machine learning (ML) approaches, run orders of magnitude faster on GPUs (Graphical Processing Units), which are built for parallel processing and can run thousands of processes simultaneously, as opposed to CPUs (Central Processing Units), which run processes sequentially.

While GPUs were initially designed for real-time rendering of video games, recent generations, designed specifically for ML have allowed them to make efficiency gains of up to 40% per year [1]. The creation of hardware such as TPUs (Tensor Processing Units) and NPUs (Neural Processing Units), which are specifically designed for ML workloads, have allowed further efficiency gains to be achieved. Generally, this class of hardware is often referred to as an AI accelerator.

While the workloads for training and deploying AI models are different, they can be run on the same hardware, with different configurations: training requires more massive parallelism, where up to hundreds of thousands of AI accelerators are connected and act as a single supercomputer. On the other hand, deployment (also known as inference) allows more agile configurations that can be scaled up and down depending on user demand, and can accommodate different batch sizes, which means that hardware is often not fully utilized. Both types of workloads can be highly volatile, spiking and dipping in unpredictable ways, which can put huge strain on the data center and power grid it’s connected to [2]

Image Source

3. Where are they located? 🌎

Data centers have existed for several decades, as the World Wide Web, email and streaming services have become increasingly ubiquitous. Until 2018, global data center capacity was only growing by a few percentage points a year; however, in the last 7 years, their growth systematically stayed in double digits [3]. According to recent estimates, there are roughly 12,000 data centers worldwide, with about 10% of them focused on AI [4].

They are very geographically concentrated - only 32 countries have data centers, and nearly half of them are in the United States. The state of Virginia has the highest density of data centers globally - it is home to almost 35% of all hyperscale data centers worldwide. There are multiple reasons for this concentration - from the cheap electricity and high-speed fiber (including undersea cables) that allow fast data transfers, to historical reasons such as early interconnection points and local tax incentives [5]. Given that ML model training specifically needs large co-located clusters, this has further exacerbated this situation, with Virginia continuing to be the preferred location for new hyperscale data center projects [6].

Image Source

⚡ ENERGY⚡

1.. Why do AI data centers use so much energy? 🪫

Much of the recent discourse surrounding data centers centers around the large amount of energy that they require, which is putting strain on energy grids in places like Virginia [7] and is even resulting in residential consumers paying more for their energy bills [8]. Broadly speaking, this is due to several factors: on the one hand, GPUs have a much higher energy draw than CPUs, with recent generations of GPUs such as NVIDIA’s Blackwell model using up to 600W of power [9], whereas the latest CPUs only use 250W [10].

Image Source

Also, since the hardware itself is increasingly concentrated - with multiple GPUs in one server rack, and hundreds of server racks in a given data center, this makes the power draw rise proportionately. While historically CPU server racks used 5-10 kW of power, in the GPU era, racks require 10-20 times more power [4]. At the level of a hyperscale data center cluster, this can translate into requirements of up to 5 and even 10 GW of power, up from 5 MW - a 2,000 fold increase in the span of a decade [4, 11].

2. Where is that energy coming from? 🔌

According to recent estimates from the International Energy Agency, 60% of the energy used to power data centers globally is generated from fossil fuels such as coal and natural gas [12]. In the United States alone, data centers are estimated to generate more than 105 million tons of CO₂e, representing 2% of US emissions in 2023 [13]. This estimate represents the emissions from the energy that data centers physically consume (i.e. location-based emissions), rather than any contract that data center operators have to buy renewable energy credits (i.e. market-based emissions)*, which are often reported by the operators [14].

Given the strain that AI data centers are placing on energy grids (since they require a large, concentrated quantity of energy in a specific location), there are different ways in which this is currently addressed, such as “offgrid” approaches (i.e. independent microgrids) that can use different types of sources than the location in which a data center is located, including hybrid systems that combine renewables, fossil fuels and battery storage [15] or fossil fuel-based systems that provide more output that the local grid can handle [16].

3. Can data centers be fully powered by renewable energy? 🌱

While it is an exaggeration that AI data centers run 24 hours a day, 365 days a year at near-maximum demand, they do keep an intense but fluctuating pattern that uses around 80% of capacity for training and 40-60% for inference [17]. This means that the amount of electricity required by data centers can swing massively from one moment to another. What this means is that either the energy source powering the data center has to be equally responsive and dynamic, or there has to be some kind of buffer between the data center and the power source that allows to compensate for potential differences in supply and demand, which can destroy equipment and cause grid instability.

While fossil fuel generators can be very responsive to fluctuations in demand, renewable energy sources - and nuclear power plants - cannot [18]. This is logical, given that it is always theoretically possible to pump more or less natural gas to generate more or less electricity, but much harder to control the output of solar panels or wind turbines, which will respond to changes in the sun’s rays or wind speed (while nuclear power plants always have a near-steady output). Whereas it is theoretically possible to compensate for this with equipment such as batteries or condensers, this can add significant costs, making these options much more expensive for builders and operators [19].

4. Will nuclear power solve the data center energy crisis? ⚛️

Nuclear power has garnered a lot of interest in the context of AI data centers, with multiple partnerships between technology companies and nuclear startups [20] and facility operators [21] being announced in recent years. Much of this work is forward-looking, on different time horizons ranging from a few years to multiple decades. Currently, nuclear energy provides approximately 9% of the world's electricity, with over half of existing capacity concentrated in the United States, China and France [22].

While new reactors are continuing to be built globally, this takes, on average, 6 to 8 years [23]; even in cases when existing decommissioned nuclear reactors are put back online (e.g. Three Mile Island [24] and Duane Arnold Energy Center [25], which are set to be revived by Microsoft and Google), this can take 3-5 years. Even in countries like France, where surplus nuclear energy is readily available, connecting new data centers to the existing electricity grid can take up to 5 years due to the permitting and construction procedures required [26].

Finally, while new technologies such as SMRs (Small Modular Reactors, a new type of nuclear reactor that is smaller in size and produces less power than traditional nuclear reactors) are also a potentially promising way to generate energy for data centers, they are also not currently viable. The first commercial SMR in North America is only set to come online in 2029 [27] and it will take years before the technology is mature enough to be widely adopted. Other alternative approaches in nuclear energy generation, such as nuclear fusion, are also active areas of research, with companies pursuing the development of this approach garnering investment from technology companies [28] - however, they are also not currently viable on a commercial scale.

💧 WATER 💧

1. Why do data centers use so much water? 🚰

An AI data center’s water footprint is comprised of three types of water use: on-site (i.e. on-premise cooling at the data center), the water used by the power plant facilities that supply power to data centers (which is particularly high for fossil fuel, nuclear, and hydroelectric power plants), and the water consumed during the manufacturing process of materials, notably AI processing chips. On a global level, the total water consumption of data centers is estimated to be around 560 billion liters per year [13].

While initially air cooling (i.e. using fans similar to those in laptops and desktops) was sufficient for traditional, CPU-centric data centers, as the demand for hyperscale data centers filled with GPUs is rising, the use of water-based cooling also rises, since water is more efficient than air to dissipate heat. For cooling, data centers specifically use freshwater, because salt can clog up the cooling circuits. Hyperscale data centers use, on average, up to 20 million liters of water per day, as much as a town of 10,000-50,000 residents [29]. Since only 3% of the Earth’s water is freshwater, and much of its aquifers are already under pressure from drought and over-consumption [30], this limits the amount of available water, and locations, for building data centers.

2. What are potential solutions that could make data center cooling use less water? ♻️

Most data centers currently use open-loop systems, which results in as much as 80% of the water input evaporating [31] and the rest discharged back into rivers and streams. While closed-circuit cooling systems (i.e. where all of the water is recycled and none of it evaporates)[33] are technically feasible, they are more costly and therefore less common. Also, more environmentally-friendly cooling approaches, such as the usage of desalination and water reclamation [32], as well as more efficient direct-to-chip cooling [34] are starting to be adopted in the industry.

When it comes to water used for chip manufacturing, this is particularly difficult to reduce given that large quantities of ultrapure water are needed to clean, etch and rinse GPU chips during the manufacturing process, with an average facility using up to 40 million liters of water a day [31]. This can put a strain on local aquifers - for instance, Taiwan, which is home to over 90% of the world’s manufacturing plants for advanced computing chips, has experienced bouts of drought in recent years, which have led the government to shut off irrigation across tens of thousands of acres of farmland in order to prioritize manufacturing [35]. However, many new chip facilities are reducing the amount of water that they use, for instance by adopting new kinds of processes or deploying reclamation projects [36].

Conclusion

AI data centers are at the forefront of AI research and practice, and their environmental impacts are far from negligible — both in terms of the water and energy required to power them, and the emissions they generate. Given the speed and scale of AI’s growth, these impacts are rising just as quickly, while many of the proposed solutions have yet to be implemented at a scale large enough to offset them. As a community, we should, at a minimum, strive for greater transparency about these impacts and be proactive in integrating mitigation measures into the next generation of data centers.

Citation:

@inproceedings{ai_data_center_primer,
  author    = {Boris Gamazaychikov and
                Sasha Luccioni},
  title     = {⚡ Power, Heat, and Intelligence ☁️ - AI Data Centers Explained 🏭},
  booktitle = {Hugging Face Blog},
  year      = {2025},
  url       = {https://huggingface.co/blog/sasha/ai-data-centers-explained}
}

References

1. Epoch AI, (2025) ‘Data on Machine Learning Hardware’. Published online at epoch.ai.
2. Retrieved from Li, Y., & Li, Y. (2025). AI Load Dynamics--A Power Electronics Perspective. arXiv preprint arXiv:2502.01647.
3. Conte, Niccolo (2025) Charted: The Growth of Global Data Center Capacity (2005-2025), Visual Capitalist
4. Stepchange (2025) Data Centers: The Hidden Backbone of Our Modern World
5. Virginia Economic Development Partnership (2025) Data Centers
6. Beckler, Hannah (2025) Virginia's data center construction boom is even bigger than you think. One company is behind most of it. Business Insider
7. Copley, Michael (2025) Data centers are booming. But there are big energy and environmental risks NPR
8. Saul, Josh et al. (2025) AI Data Centers Are Sending Power Bills Soaring, Bloomberg Technology
9. Tech PowerUp (2025) NVIDIA RTX PRO 6000 Blackwell
10. Intel (2025) Intel® Core™ i9 processor
11. Skidmore, Zachary (2025) Fermi America files for IPO amid plans for 11GW data center campus in Texas, Data Center Dynamics
12. International Energy Agency (2025), Energy and AI
13. Guidi, G., Dominici, F., Gilmour, J., Butler, K., Bell, E., Delaney, S., & Bargagli-Stoffi, F. J. (2024). Environmental burden of United States data centers in the artificial intelligence era. arXiv preprint arXiv:2411.09786.
14. O’Brien, Isabel (2024) Data center emissions probably 662% higher than big tech claims. Can it keep up the ruse?, The Guardian
15. Rollinson, William (2025) Off-Grid Microgrids: The Future of Sustainable Data Centres, Data Center Magazine
16. Gooding, Matthew (2025) xAI granted permits for 15 gas turbines at Memphis data center, Data Center Dynamics
17. Norris, Tyler (2025), The Puzzle of Low Data Center Utilization Rates, Power Policy Net
18. Kirby, B., & Milligan, M. R. (2005). A method and case study for estimating the ramping capability of a control area or balancing authority and implications for moderate or high wind penetration. WINDPOWER 2005 Conference
19. Eliahou Ontiveros, Jeremie et al (2025) AI Training Load Fluctuations at Gigawatt-scale - Risk of Power Grid Blackout? Semi Analysis
20. Massa, Annie (2024) Altman’s $3.7 Billion Fusion Startup Leaves Scientists Puzzled, Bloomberg Law
21. Kearney, Laila and Srivastava, Vallari (2025) NextEra Energy partners with Google to restart Iowa nuclear plant, Reuters
22. World Nuclear Association (2025), Nuclear Power in the World Today
23. Ritchie, Hannah (2023), How long does it take to build a nuclear reactor? Sustainability by Numbers
24. Mandler, C. (2024) Three Mile Island nuclear plant will reopen to power Microsoft data centers, NPR
25. Butts, Dylan (2025) Google and NextEra to revive major Iowa nuclear facility as AI energy demand surges, CNBC
26. Crellin Forrest and Howcroft Elizabeth (2025), Connection challenge could hamper nuclear powered France's bid as AI hub, Reuters
27. OPG (2025), OPG ready to begin building North America’s first Small Modular Reactor
28. Terrell, Michael (2025), Our latest bet on a fusion-powered future, Google
29. Yañez-Barnuevo, Miguel (2025), Data Centers and Water Consumption, Environmental and Energy Study Institute
30. National Geographic (2025), Earth's Freshwater
31. Li, Pengfei, Jianyi Yang, Mohammad A. Islam, and Shaolei Ren. (2025) "Making AI less ' thirsty'." Communications of the ACM 68, no. 7, 54-61.
32. Setmajer, Alex (2025) How Data Centers Can Use Alternative Water Sources, Equinix Blog
33. Tozzi, Christopher (2024), What Is Closed-Loop Cooling, and When Should Data Centers Use It? Data Center Knowledge
34. Hamilton, Marc (2025) Chill Factor: NVIDIA Blackwell Platform Boosts Water Efficiency by Over 300x, NVIDIA Blog
35. Zhong, Raymond and Chang Chien, Amy (2021) Drought in Taiwan Pits Chip Makers Against Farmers, The New York Times
36. Wiles, Russ (2023) Taiwan Semiconductor Manufacturing Co. nearing manufacturing start date in north Phoenix, AZ Central

Footnotes

*Renewable energy credits are a form of carbon offsetting that are often used by companies to meet their net-zero commitments - see https://www.ibm.com/think/topics/renewable-energy-certificates