  Both the training and use of Large Language Models (LLMs) require large
amounts of energy. Their increasing popularity, therefore, raises critical
concerns regarding the energy efficiency and sustainability of data centers
that host them. This paper addresses the challenge of reducing energy
consumption in data centers running LLMs. We propose a hybrid data center model
that uses a cost-based scheduling framework to dynamically allocate LLM tasks
across hardware accelerators that differ in their energy efficiencies and
computational capabilities. Specifically, our workload-aware strategy
determines whether tasks are processed on energy-efficient processors or
high-performance GPUs based on the number of input and output tokens in a
query. Our analysis of a representative LLM dataset, finds that this hybrid
strategy can reduce CPU+GPU energy consumption by 7.5% compared to a
workload-unaware baseline.
