Every server, switch, and storage drive in a data center does the same thing: it turns electricity into computation — and heat. Lots of it. A single high-density rack can dissipate up to 30–50 kilowatts of thermal energy. An entire hyperscale facility holding thousands of racks? We’re talking tens of megawatts of waste heat that must be continuously removed or the whole thing melts down — sometimes literally.
This is not a niche engineering problem. Cooling accounts for roughly 40% of a data center’s total energy consumption. As AI workloads push GPU clusters to their thermal limits, the battle against heat has become one of the defining challenges of our digital age.
Before diving into specific technologies, it helps to understand the three fundamental mechanisms all cooling systems exploit.
Conduction is the simplest: heat flows from hot to cold through direct contact. Inside a server, copper heat spreaders, thermal paste, and heatsinks all move heat away from processor dies via conduction. Convection takes over next — a moving fluid, whether air or liquid, carries that heat away from the heatsink and out of the chassis. And evaporation is the secret weapon of the most efficient systems: when a liquid vaporizes, it absorbs an enormous amount of energy in the process, which is why sweating cools you down and why cooling towers produce that characteristic mist.
Every data center cooling system is, at its core, a combination of these three mechanisms arranged in a chain that ends with heat being rejected to the outdoor environment.
For decades, air was the default medium for data center cooling. Two major variants dominate this landscape: the Computer Room Air Conditioner (CRAC) and the Computer Room Air Handler (CRAH).
A CRAC unit is essentially a self-contained refrigeration system — like a precision-engineered air conditioner bolted to the data center floor. It uses a compressor-driven refrigerant cycle: warm air from the hot aisle is drawn in, passed over evaporator coils filled with refrigerant, cooled, and blown back out into the cold aisle at around 18–21°C. CRAC units are self-sufficient and can operate without any external chilled water infrastructure, making them attractive for smaller deployments. The downside is energy consumption — the compressor is power-hungry, and efficiency drops in warmer ambient conditions.
A CRAH unit takes a different approach. Instead of generating its own cold, it circulates chilled water supplied by a central chiller plant located outside the building. Warm room air passes over the chilled water coils, transfers heat to the water, and exits cool. The now-warmed water returns to the chiller to be cooled again. The tradeoff is infrastructure dependency — you need a chiller plant, cooling towers, and extensive piping — but the reward is dramatically better efficiency at scale.
Whether using CRAC or CRAH units, operators deploy hot aisle/cold aisle containment to maximize airflow efficiency. Server racks are arranged in alternating rows: one row faces forward into the cold aisle where cool air enters, the next faces backward into the hot aisle where exhaust heat collects. Physical containment systems — curtains, doors, or ceiling panels — prevent hot and cold air from mixing before the heat is removed. Properly implemented containment can reduce cooling energy consumption by 20–40% compared to open-floor layouts. The math is simple: if air that has already mixed to 30°C arrives at the CRAC, the unit works twice as hard as if pure 45°C exhaust air arrived.
In large facilities, CRAH units connect to a central cooling plant — a hierarchy of equipment that ultimately rejects heat to the outdoor environment.
The chiller is the heart of this system. Industrial chillers work on the same vapor-compression refrigeration cycle as a CRAC unit, but at massive scale — units rated at hundreds or thousands of tons of cooling capacity. They chill a closed loop of water that feeds CRAH units throughout the building.
That heat has to go somewhere. Cooling towers are the final step: warm water from the chiller’s condenser side is pumped to the roof or exterior, where it’s sprayed into a tower and exposed to airflow. Evaporation carries away the heat, which is why you see those tall structures with mist rising from them near large facilities. A typical cooling tower loses 1–3% of its water volume per hour to evaporation, which is why data centers near water-scarce regions face serious sustainability challenges.
As rack densities climbed through the 2010s, traditional perimeter CRAC units struggled to push cold air to the center of large data halls. The solution was to bring cooling physically closer to the heat source.
In-row coolers are CRAH-style units designed to fit between server racks in the same row. They create short, tight airflow loops, pulling hot air from the adjacent hot aisle and returning cold air directly into the servers. Because the distances are short and mixing is minimal, they scale gracefully to higher densities than perimeter units can handle.
For the highest-density deployments, cooling units can be integrated directly into the rack enclosure itself. A rear-door heat exchanger sits at the back of the rack like a radiator, capturing exhaust heat before it ever reaches the room environment. Some designs achieve near-total containment — the room sees essentially no thermal load from the rack at all.
Air has a fundamental limitation: it is a terrible heat transfer medium. Liquid water carries roughly 3,500 times more heat per unit volume than air. As AI GPUs began dissipating 300–700 watts per chip — with entire server nodes cramming eight or more chips — air cooling simply could not keep up. The industry pivoted to liquid.
The most common form of liquid cooling today routes chilled water through a metal cold plate — usually copper or aluminum — that sits directly on top of the processor, GPU, or memory module. The cold plate connects to a facility water loop via quick-disconnect fittings built into the server chassis. Heated water exits into a Cooling Distribution Unit (CDU), a rack-scale device that manages pressure, flow rate, and temperature for a cluster of servers. The CDU connects to the facility’s primary water loop, where a heat exchanger transfers the heat out before returning cool water to the servers.
A key advantage of modern warm-water cooling is that the incoming water does not need to be chilled to frigid temperatures. Many systems operate effectively with 30–45°C inlet water — warm enough to skip the energy-intensive chiller entirely and reject heat directly to the outdoor environment or even capture it for building heating.
If cold plates still are not sufficient — and for bleeding-edge AI training clusters, they sometimes are not — operators turn to immersion cooling, where the servers themselves are submerged in liquid.
Single-phase immersion uses a non-conductive dielectric fluid, such as synthetic hydrocarbons or engineered fluorocarbons, that remains liquid as it absorbs heat. Servers sit in open bath tanks. The fluid circulates by convection or pumps, and heat is extracted via a heat exchanger at the tank’s top. Hardware remains accessible, and the fluid can be reused indefinitely with filtration.
Two-phase immersion goes further by using a fluid with a very low boiling point — around 40–50°C. As components heat the fluid, it vaporizes, rises to a condenser coil at the top of the tank, liquefies, and rains back down. This phase-change cycle carries enormous amounts of heat with zero pumping energy for the fluid itself. It is the most efficient cooling method available today, but also the most complex to operate at scale.
Power Usage Effectiveness (PUE) is the industry-standard metric for data center efficiency. The formula is straightforward: divide total facility power by IT equipment power. A PUE of 1.0 is theoretical perfection — every watt goes to computation, none to cooling or power distribution. A PUE of 2.0 means for every watt of compute, another full watt is consumed by infrastructure.
The global average sits around 1.58. Hyperscale operators like Google, Meta, and Microsoft routinely achieve 1.1–1.2. The most efficient immersion-cooled facilities have reached below 1.05. Cooling technology is the single biggest lever for moving this number.
Perhaps the biggest efficiency leap of the past decade is the adoption of “free cooling” or economizer modes — operating periods where the outdoor environment is cold enough to reject data center heat without running energy-intensive chillers at all.
In cooler climates — northern Europe, the Pacific Northwest, Scandinavia — data centers can run in free-cooling mode for thousands of hours per year. The warm return water from servers is simply pumped through a dry cooler exposed to cold outdoor air, cooled, and returned; the chiller sits idle. This is why such concentrations of data centers exist in places like Dublin, Stockholm, and the Columbia River Gorge.
Warm-water liquid cooling amplifies this advantage dramatically. If servers are happy with 40°C inlet water, you can reject heat without chillers on any day when the outdoor temperature is below roughly 35°C — that is most of the year in most inhabited places on Earth.
The next generation of AI accelerators will push single-chip power dissipation above 1,000 watts. Racks will hit 150–500 kW of density. Air cooling is simply not a viable path at these power levels; the physics do not allow it.
The industry is converging on a hybrid model: liquid cooling handles the high-power compute — GPUs, TPUs, custom AI chips — while residual heat from storage, networking, and memory is managed by air. Fully liquid-cooled facilities optimized for AI inference and training are already under construction globally.
Waste heat recovery is an emerging area of enormous potential. Several facilities in Europe now capture data center exhaust heat to warm homes and offices, turning a liability into a resource. A warm-water liquid cooling system running at 50°C outlet can directly feed district heating networks. As energy costs and sustainability pressures mount, this may shift from novelty to norm.
