  By leveraging the underlying structures of the quadrotor dynamics, we propose
multi-agent reinforcement learning frameworks to innovate the low-level control
of a quadrotor, where independent agents operate cooperatively to achieve a
common goal. While single-agent reinforcement learning has been successfully
applied in quadrotor controls, training a large monolithic network is often
data-intensive and time-consuming. Moreover, achieving agile yawing control
remains a significant challenge due to the strongly coupled nature of the
quadrotor dynamics. To address this, we decompose the quadrotor dynamics into
translational and yawing components and assign collaborative reinforcement
learning agents to each part to facilitate more efficient training.
Additionally, we introduce regularization terms to mitigate steady-state errors
and prevent excessive maneuvers. Benchmark studies, including sim-to-sim
transfer verification, demonstrate that our proposed training schemes
substantially improve the convergence rate of training, while enhancing flight
control performance and stability compared to traditional single-agent
approaches.
