  This study focuses on designing and developing a mathematically based
quadcopter rotational dynamics simulation framework for testing reinforcement
learning (RL) algorithms in many flexible configurations. The design of the
simulation framework aims to simulate both linear and nonlinear representations
of a quadcopter by solving initial value problems for ordinary differential
equation (ODE) systems. In addition, the simulation environment is capable of
making the simulation deterministic/stochastic by adding random Gaussian noise
in the forms of process and measurement noises. In order to ensure that the
scope of this simulation environment is not limited only with our own RL
algorithms, the simulation environment has been expanded to be compatible with
the OpenAI Gym toolkit. The framework also supports multiprocessing
capabilities to run simulation environments simultaneously in parallel. To test
these capabilities, many state-of-the-art deep RL algorithms were trained in
this simulation framework and the results were compared in detail.
