  This paper presents Gym-Ignition, a new framework to create reproducible
robotic environments for reinforcement learning research. It interfaces with
the new generation of Gazebo, part of the Ignition Robotics suite, which
provides three main improvements for reinforcement learning applications
compared to the alternatives: 1) the modular architecture enables using the
simulator as a C++ library, simplifying the interconnection with external
software; 2) multiple physics and rendering engines are supported as plugins,
simplifying their selection during the execution; 3) the new distributed
simulation capability allows simulating complex scenarios while sharing the
load on multiple workers and machines. The core of Gym-Ignition is a component
that contains the Ignition Gazebo simulator and exposes a simple interface for
its configuration and execution. We provide a Python package that allows
developers to create robotic environments simulated in Ignition Gazebo.
Environments expose the common OpenAI Gym interface, making them compatible
out-of-the-box with third-party frameworks containing reinforcement learning
algorithms. Simulations can be executed in both headless and GUI mode, the
physics engine can run in accelerated mode, and instances can be parallelized.
Furthermore, the Gym-Ignition software architecture provides abstraction of the
Robot and the Task, making environments agnostic on the specific runtime. This
abstraction allows their execution also in a real-time setting on actual
robotic platforms, even if driven by different middlewares.
