Artificial intelligence is reshaping the physical design of data centers. The rapid expansion of large-scale model training, generative-AI inference, high-performance computing, and GPU-accelerated platforms is increasing both electrical demand and thermal density. Cooling can no longer be treated as a secondary facility service. It has become a defining constraint on how AI infrastructure is designed, deployed, operated, and scaled.
The International Energy Agency projects that global electricity consumption from data centers could rise from approximately 460 TWh in 2024 to more than 1,000 TWh by 2030 in its base case. AI-focused facilities are a major contributor to this growth, although the projection covers the full data-center sector rather than AI alone. (IEA)
The central challenge is not simply that AI consumes more electricity. It is that compute power is becoming concentrated in fewer, denser systems. This concentration changes the thermal physics of the data center and increasingly requires liquid cooling, hybrid air-liquid architectures, advanced coolant distribution units, and integrated thermal design from processor to heat-rejection equipment.
1. Compute Density Is Redefining Thermal Design
Earlier generations of data centers were largely designed around air-cooled servers, with rack densities commonly below 20 kW. Today, modern accelerated-computing platforms have moved well beyond that operating model.
For example, NVIDIA’s H200 GPU has a configurable thermal design power of up to 700 W. AMD’s MI355X reaches 1,400 W of typical board power in its direct-liquid-cooled configuration. At the system level, NVIDIA lists the DGX B200 at approximately 14.3 kW maximum system power, while an NVIDIA GB200 NVL72 rack is specified at approximately 120 kW and uses liquid cooling. (NVIDIA)
These figures demonstrate why AI cooling cannot be addressed through incremental changes to traditional room-air systems. The relevant design unit is no longer merely the server room or the hot aisle. It is the integrated thermal path extending from the chip, through the server and rack, into the facility water loop, and finally to the external heat-rejection system.
2. The Limits of Air Cooling
Air cooling remains effective for many conventional IT workloads, and it can be extended through containment, in-row cooling, rear-door heat exchangers, optimized airflow management, and high-efficiency air-handling systems.
However, air has a relatively low volumetric heat capacity. Removing large thermal loads through air therefore requires very high airflow volumes, powerful fans, substantial plenum capacity, strict containment, and increasingly complex mechanical infrastructure.
There is no universal rack-density threshold at which air cooling fails, because the answer depends on server design, ambient conditions, airflow paths, allowable temperatures, redundancy requirements, and facility architecture. Nevertheless, industry guidance increasingly identifies 80–100 kW per rack as a range in which air-based systems become inadequate for many AI deployments and liquid cooling becomes the preferred engineering approach. (Schneider Electric)
The operational implication is important: the transition to liquid cooling is not driven by fashion or technology marketing. It is driven by the thermal concentration of modern accelerators and the limits of moving heat through air at scale.
3. Liquid Cooling as a Structural Architectural Change
Liquid cooling changes the thermal architecture of the data center because liquids can transport much more heat than air within a compact physical volume.
Two principal approaches are currently used:
Direct-to-Chip Cooling
Direct-to-chip, also called direct liquid cooling, circulates coolant through cold plates mounted directly on high-power components such as CPUs, GPUs, accelerators, and high-bandwidth memory modules.
The liquid captures heat close to the silicon and transports it away through dedicated hydraulic circuits. Residual heat from components such as storage, networking, power supplies, and memory may still be handled by air cooling. This makes direct-to-chip architectures especially suitable for hybrid deployments and for retrofitting existing facilities. (Schneider Electric)
Direct-to-chip cooling has become the leading practical route for many enterprise AI and high-density deployments because it can support high thermal loads without requiring the complete immersion of server hardware. (Schneider Electric)
Immersion Cooling
Immersion cooling places servers, or major server components, inside dielectric fluids designed to absorb heat directly from the hardware.
This approach can support very high thermal densities, but it requires specialized hardware compatibility, fluid-management procedures, maintenance practices, materials validation, and lifecycle planning. It is therefore often used in more specialized environments rather than as the default migration path for all existing data centers. (ashrae.org)
4. The Chip-to-Chiller Thermal Architecture
AI data centers are increasingly designed around a chip-to-chiller architecture. This is not a single universal standard, but a useful model for understanding how heat moves through a modern liquid-cooled system.
The process can be understood in six stages:
- Silicon heat generation: CPUs, GPUs, memory, networking devices, and power electronics generate thermal load.
- Cold plates: Liquid-cooled plates extract heat from the highest-power components.
- Technology Cooling System: The IT-side loop transports heated coolant from the servers and racks.
- Coolant Distribution Unit: The CDU manages the interface between the IT cooling loop and the building-side cooling system.
- Facility Water System: The facility loop transports heat away from the IT equipment.
- Heat rejection: Dry coolers, chillers, cooling towers, or economization systems reject the heat to the external environment.
A critical design principle is the separation between the IT cooling loop and the facility water loop. The CDU uses a heat exchanger to transfer heat while preserving hydraulic and chemical separation between the two circuits. This protects sensitive server-side equipment from pressure fluctuations, contamination risks, and unstable water conditions in the broader facility system. (Schneider Electric Blog)
5. Why the Coolant Distribution Unit Is Central
The coolant distribution unit is not an accessory. It is the hydraulic and thermal control center of a liquid-cooled AI environment.
A properly designed CDU typically provides:
- Heat exchange between the IT loop and facility loop.
- Flow control for liquid-cooled servers and racks.
- Temperature management.
- Pressure control and protection.
- Fluid isolation and treatment support.
- Monitoring and integration with facility-management systems.
Without stable flow and pressure control, cold plates can experience insufficient coolant delivery, uneven thermal performance, leakage risk, or unstable operating temperatures. Schneider Electric identifies the CDU as the primary interface between the IT fluid loop and facility cooling loop, with responsibilities for temperature, flow, pressure, fluid treatment, heat exchange, and isolation. (Schneider Electric Blog)
For this reason, CDU selection should be based on more than nominal capacity. Operators must assess redundancy, maintainability, scalability, controls integration, pressure envelope, fluid compatibility, and failure-response requirements.
6. Grid-to-Chip and Chip-to-Chiller Must Be Designed Together
High-density AI facilities require the simultaneous design of two interdependent chains.
The first is grid-to-chip: the electrical path from the grid, transformers, switchgear, UPS systems, busways, power shelves, and rack distribution to the processors.
The second is chip-to-chiller: the thermal path from the processor to the cold plate, CDU, facility water circuit, and heat-rejection equipment.
These two systems cannot be engineered independently. A 100 kW or 120 kW rack affects electrical distribution, floor loading, cable routing, coolant piping, pump capacity, heat-rejection equipment, structural requirements, fire-safety planning, maintenance access, and commissioning procedures.
ASHRAE highlights that high-density AI facilities require integrated planning for liquid cooling, major power availability, networking, and the structural demands of heavier and denser equipment. (ashrae.org)
The result is a change in engineering philosophy: cooling is no longer simply room infrastructure. It becomes compute-adjacent thermal engineering.
7. Heat Rejection, Water Use, and Energy Efficiency
Liquid cooling can improve thermal efficiency because it can move heat at higher temperatures and reduce dependence on high-volume airflow and compressor-intensive cooling.
However, liquid cooling does not automatically guarantee lower energy consumption or lower water use. Outcomes depend on climate, water availability, cooling topology, supply-water temperature, economizer hours, chiller design, workload profile, and the operational setpoints chosen by the operator.
Water-side economization can reduce mechanical cooling by using favorable outdoor conditions to cool facility water indirectly. However, its effectiveness depends on local climate and system design. Some regions may be better suited to dry coolers, while others may require hybrid or chiller-assisted solutions to meet reliability and temperature requirements. (ashrae.org)
A rigorous AI-data-center strategy should therefore evaluate at least four performance dimensions:
- IT capacity: how much compute power can be supported reliably.
- Energy efficiency: how much facility energy is required per unit of IT load.
- Water efficiency: how much water is consumed or withdrawn under normal and peak conditions.
- Thermal resilience: whether the infrastructure remains stable during outages, heat waves, maintenance events, and rapid workload changes.
8. Operational Risks and Design Controls
Liquid cooling introduces new operational requirements. These do not make the technology unsafe or impractical, but they require disciplined engineering and operations.
The critical areas include:
- Water chemistry and fluid-quality management.
- Leak detection and containment.
- Pressure monitoring and relief mechanisms.
- Redundant pumping and CDU capacity.
- Flow balancing across racks and cold plates.
- Quick-disconnect reliability.
- Commissioning procedures.
- Telemetry integration with BMS, DCIM, and IT management platforms.
- Vendor interoperability and serviceability.
Schneider Electric’s guidance on direct liquid cooling emphasizes that specification, installation, and operation introduce distinct challenges that must be addressed early in the design process rather than after deployment. (Schneider Electric)
The practical lesson is straightforward: liquid cooling should be designed as a controlled system, not installed as an isolated hardware upgrade.
9. Brownfield and Greenfield Deployment Strategies
Brownfield Facilities
Existing air-cooled data centers can often be adapted through a staged migration strategy:
- Identify the racks and workloads with the highest thermal density.
- Validate structural, electrical, and hydraulic capacity.
- Add direct-to-chip loops, CDUs, and rear-door heat exchangers where needed.
- Maintain air cooling for residual server heat.
- Expand liquid infrastructure in phases as AI capacity grows.
This hybrid approach can support AI deployments without immediately rebuilding the entire facility. (Schneider Electric)
Greenfield Facilities
New AI-focused data centers should be designed from the outset for liquid readiness. This includes facility water loops, CDU placement, manifold access, pipe routing, leak detection, drainage, high-capacity electrical distribution, scalable heat rejection, and space for maintenance and future density growth.
A greenfield design should not optimize only for today’s accelerator generation. It should preserve enough thermal, electrical, and mechanical flexibility to support future chip power and rack-density increases.
10. Cooling as a Capacity Multiplier
Liquid cooling should not be evaluated only as an efficiency project. It is also a capacity-enablement strategy.
By improving heat removal at the chip and rack level, it allows operators to deploy more accelerators in a given footprint, reduce thermal throttling risk, improve hardware stability, and support rack-scale AI systems that traditional air cooling cannot accommodate efficiently.
This matters in an AI market where advanced accelerators, high-bandwidth memory, power infrastructure, grid interconnections, and data-center capacity all remain strategic constraints.
The true value of liquid cooling is therefore not merely that it removes heat. It allows data centers to convert electrical power, floor space, and scarce accelerated hardware into usable computational capacity.
11. Conclusion
Artificial intelligence is redefining the thermal limits of digital infrastructure.
The transition from conventional air cooling to liquid and hybrid cooling architectures is being driven by the concentration of compute power in modern GPU and accelerator platforms. Systems such as the NVIDIA GB200 NVL72, with approximately 120 kW rack power and liquid cooling, illustrate that AI infrastructure is increasingly designed at rack scale rather than server scale. (NVIDIA Docs)
Direct-to-chip liquid cooling is emerging as the most practical mainstream architecture for many enterprise and hyperscale deployments because it can support high-density compute while preserving a manageable migration path from air-cooled infrastructure. (Schneider Electric)
The next generation of AI data centers will be defined not only by processor performance, model scale, or electrical capacity. It will be defined by the quality of their thermal architecture: the ability to coordinate grid-to-chip power delivery with chip-to-chiller heat removal, maintain reliable hydraulic control, manage energy and water responsibly, and scale without compromising availability.