New generation of AI may drastically increase data center energy consumption, study points out

Energy Impact of the Research

The investigation demonstrated that energy expenditure can be up to 136.5 times higher than recorded in conventional question-and-answer systems. This increase occurs because an AI agent needs to repeatedly consult LLMs during the execution of a single task, which demands much more intensive processing. Key findings include energy consumption up to 136.5 times greater per query, an increase in latency of up to 153.7 times, and the fact that GPUs remain idle for up to 54.5% of the execution time. Furthermore, an average consumption of 348.41 watt-hours per query was recorded in a model with 70 billion parameters.

This pioneering study offers a clear quantification not only of how AI is becoming more sophisticated but also of how much electricity and what costs are inherent in the implementation and maintenance of this intelligence. Minsoo Rhu, a professor at the KAIST School of Electrical Engineering, commented on the results.

Future Projections and Challenges

The researchers also simulated a hypothetical scenario involving 13.7 billion daily requests from AI agents, a volume comparable to current research conducted at Google. In this projection, data centers would consume approximately 198.9 gigawatts, equivalent to about half of the average energy consumption of the United States. The study also pointed out another critical point: while external tools perform certain steps, high-performance GPUs remain inactive for part of the time, which reduces the overall system efficiency.

For the KAIST team, the progress of AI will depend on a coordinated evolution among models, AI chips, data centers, and energy infrastructure. Minsoo Rhu emphasized the need to adopt an integrated co-design strategy to optimize both the data center infrastructure for AI and the AI agent models and the power grid as AI agents proliferate.

Additionally, the researchers made the implementations of the AI agents and the benchmarks used in the study available in open source, which should drive new research and aid in developing a more sustainable infrastructure.