  In this paper, we investigate the amalgamation of cell-free (CF) and
extremely large-scale multiple-input multiple-output (XL-MIMO) technologies,
referred to as a CF XL-MIMO, as a promising advancement for enabling future
mobile networks. To address the computational complexity and communication
power consumption associated with conventional centralized optimization, we
focus on user-centric dynamic networks in which each user is served by an
adaptive subset of access points (AP) rather than all of them. We begin our
research by analyzing a joint resource allocation problem for energy-efficient
CF XL-MIMO systems, encompassing cooperative clustering and power control
design, where all clusters are adaptively adjustable. Then, we propose an
innovative double-layer multi-agent reinforcement learning (MARL)-based scheme,
which offers an effective strategy to tackle the challenges of high-dimensional
signal processing. In the section of numerical results, we compare various
algorithms with different network architectures. These comparisons reveal that
the proposed MARL-based cooperative architecture can effectively strike a
balance between system performance and communication overhead, thereby
improving energy efficiency performance. It is important to note that
increasing the number of user equipments participating in information sharing
can effectively enhance SE performance, which also leads to an increase in
power consumption, resulting in a non-trivial trade-off between the number of
participants and EE performance.
