  5G radio access network (RAN) is consuming much more energy than legacy RAN
due to the denser deployments of gNodeBs (gNBs) and higher single-gNB power
consumption. In an effort to achieve an energy-conserving RAN, this paper
develops a dynamic on-off switching paradigm, where the ON/OFF states of gNBs
can be dynamically configured according to the evolvements of the associated
users. We formulate the dynamic sleep control for a cluster of gNBs as a Markov
decision process (MDP) and analyze various switching policies to reduce the
energy expenditure. The optimal policy of the MDP that minimizes the energy
expenditure can be derived from dynamic programming, but the computation is
expensive. To circumvent this issue, this paper puts forth a greedy policy and
an index policy for gNB sleep control. When there is no constraint on the
number of gNBs that can be turned off, we prove the dual-threshold structure of
the greedy policy and analyze its connections with the optimal policy. Inspired
by the dual-threshold structure and Whittle index, we develop an index policy
by decoupling the original MDP into multiple one-dimensional MDPs -- the
indexability of the decoupled MDP is proven and an algorithm to compute the
index is proposed. Extensive simulation results verify that the index policy
exhibits close-to-optimal performance in terms of the energy expenditure of the
gNB cluster. As far as the computational complexity is concerned, on the other
hand, the index policy is much more efficient than the optimal policy, which is
computationally prohibitive when the number of gNBs is large.
