  The highly heterogeneous ecosystem of Next Generation (NextG) wireless
communication systems calls for novel networking paradigms where
functionalities and operations can be dynamically and optimally reconfigured in
real time to adapt to changing traffic conditions and satisfy stringent and
diverse Quality of Service (QoS) demands. Open Radio Access Network (RAN)
technologies, and specifically those being standardized by the O-RAN Alliance,
make it possible to integrate network intelligence into the once monolithic RAN
via intelligent applications, namely, xApps and rApps. These applications
enable flexible control of the network resources and functionalities, network
management, and orchestration through data-driven control loops. Despite recent
work demonstrating the effectiveness of Deep Reinforcement Learning (DRL) in
controlling O-RAN systems, how to design these solutions in a way that does not
create conflicts and unfair resource allocation policies is still an open
challenge. In this paper, we perform a comparative analysis where we dissect
the impact of different DRL-based xApp designs on network performance.
Specifically, we benchmark 12 different xApps that embed DRL agents trained
using different reward functions, with different action spaces and with the
ability to hierarchically control different network parameters. We prototype
and evaluate these xApps on Colosseum, the world's largest O-RAN-compliant
wireless network emulator with hardware-in-the-loop. We share the lessons
learned and discuss our experimental results, which demonstrate how certain
design choices deliver the highest performance while others might result in a
competitive behavior between different classes of traffic with similar
objectives.
