What are the impacts of artificial intelligence on data centers?

In 1950, Alan Turing asked a fundamental question: "Can machines think?". He proposed the "Turing test," an experiment designed to determine whether a machine can demonstrate human intelligence and not be distinguished from a human in the responses provided to a given question. This experiment laid the foundations for reflection on AI. However, the following decades saw slow progress, notably due to insufficient technical means.

In the 90s, AI experienced a new boom thanks to computer progress, the advent of the Internet, and Big Data, which allowed for extensive collaboration and the production of the volumes of data necessary to train algorithms effectively. Since then, AI has become a key tool in the 4.0 era with applied research in fields as diverse as research, healthcare, finance, office automation, industrial modeling, graphics, translation, computer coding, etc.

Machine learning and deep learning aim to create systems capable of solving problems autonomously, based on the principles of rationality and the imitation of human intelligence.

Today, the AI market has entered a very active growth phase with the release of ChatGPT 3.5 in 2022, a so-called generative AI, applying base models that allow for the creation of original content (text, images, structures, etc.). Since its launch, several increasingly powerful generative AI tools have emerged, supported by considerable financial resources.

To process large amounts of data quickly in order to solve complex problems such as generating text, images, or even a 2D or 3D plan, AI requires servers with specific powerful processors. Indeed, in data centers, the traditional CPU (Central Processing Unit) is being replaced by the GPU (Graphic Processing Unit), which in terms of calculation and operation is 100 times more powerful than the CPU for AI uses, or by the TPU (Tensor Processing Unit), an integrated circuit specifically developed by Google to accelerate AI systems. The TPU is approximately 2.5 times more powerful than the GPUs currently deployed in GAFAM supercomputers (HPC). It is estimated that several thousand or even tens of thousands of GPUs are needed to train the largest AI models like GPT-3 or GPT-4.

The race for power is just beginning: the new Blackwell chip unveiled on March 18, 2024, by NVIDIA, aims to support the growth of AI with capacities never before equaled.

These new chips embedded in servers to run AI technologies radically change the way we think about, design, and build, but above all, cool and power data centers.

Thus, current data centers must be adapted both in terms of their design and their operation to meet the requirements of AI. Stakeholders in the data center sector have anticipated and are adapting to these new requirements.

Rack density—the power consumed by a server cabinet—represents one of the fundamental differences between traditional data centers and those dedicated to AI. Multi-purpose data centers have average rack power densities of 5 to 15 kilowatts (kW), while those for AI are ten times higher and can exceed 100 kW per rack. With such power per rack, AI cannot be operated in a "classic" data center because the air exchange capacity is not sufficient to dissipate such heat power densities.

New, more efficient cooling systems are necessary, notably direct liquid cooling (DLC), generally using water, or immersion cooling, which involves immersing servers in oil baths to dissipate the heat produced. It should be noted that water circulates in a closed loop, which results in very low water consumption, limited to filling the cooling circuit. Many existing or planned French data centers are being adapted for these cooling techniques. Other avenues such as gas or liquid metal DLC cooling are also being studied.

The infrastructures used for training AI models do not have the same requirements in terms of service continuity as the infrastructures used in inference data processing centers. The trend that seems to be emerging is rather that of hybridization: performing training in a denser and smaller data center and inference in a less dense or even classic data center.

In the age of AI, data centers require more flexible, modular, and adaptive infrastructures. In a word, hybrid infrastructures.

This could also involve bringing AI processing as close as possible to users, particularly with SLMs (Small Language Models) on smartphones. Meshed Edge data centers distributed across territories allow machines to be brought closer to users by offloading hyperscale data centers and reducing latency.

The data center industrial sector is particularly committed to reducing energy consumption. The average PUE (Power Usage Effectiveness—the benchmark energy performance indicator; the closer it is to 1, the more efficient the data center) of data centers went from 2.5 in 2005 to 1.58 in 2023 [Source: average PUE]. The acceptability of these data centers depends on the preservation of electrical resources for all users. A close partnership with the French operator RTE is therefore necessary to balance and anticipate needs. According to a recent study by Ademe, data centers accounted for only 16% (compared to 79% for terminals) of the carbon impact of digital technology in France in 2022.

According to a study by the Uptime Institute, the majority of current racks for (generative) AI training and inference are racks that consume less than 20 kW. This means that generative AI (20% of server and infrastructure uses and utilization worldwide) is not currently synonymous with extreme densities and therefore huge electricity needs.

However, it is important to anticipate strong potential growth in future needs. Indeed, the IEA predicts that energy demand from artificial intelligence and cryptocurrencies could double between 2026 and 2030, or even more according to certain prospective scenarios. This inevitably raises the question of the additional production of this energy and its impact on the environment and society as a whole.

To control and reduce these problematic consumption projections, many avenues are already being discussed. To name just a few:

• Peak shaving to provide flexibility to the grid (data centers are generally equipped with autonomous production solutions such as biofuel or gas generators).
• Trigeneration, a technology that allows for the production of heat, cooling, and electricity from natural gas.
• Free cooling, which drastically limits consumption for the production of the cooling necessary to cool servers.
• Waste heat recovery by users who need it, which de facto limits their energy consumption.
• Shifting data processing from one data center to another in a different geographical area to reduce local electricity consumption (flexibility enabled by the cloud);
• The potential, and questionable, use of small modular reactors (SMRs) providing up to 300 MW of energy.
• Bringing data processing closer to users with SLMs.

Although AI leads to rapidly growing energy consumption, it is already generating significant progress in efficiency and productivity. In a key sector like clinical research, AI enabled the extremely rapid development of the COVID-19 vaccine.

Without the computing and processing power of data centers, researchers would not have succeeded in processing and analyzing billions of data points in record time. It is thus thanks to AI (and data centers) that vaccines were able to be developed and brought to market so quickly during the COVID-19 health crisis.

The potential of AI is enormous, and the challenges related to its use are just beginning. All stakeholders are concerned: data center players as well as end users. The former must continue their efforts to reduce energy consumption and environmental impact through the innovation that has always characterized the sector. Meanwhile, the latter are stakeholders and are responsible for the increase in consumption. It is therefore essential that everyone seeks a sober and responsible use of the high-performance services offered by AI on a daily basis.