  We propose a novel framework for Deep Reinforcement Learning (DRL) in modular
robotics to train a robot directly from joint states, using traditional robotic
tools. We use an state-of-the-art implementation of the Proximal Policy
Optimization, Trust Region Policy Optimization and Actor-Critic
Kronecker-Factored Trust Region algorithms to learn policies in four different
Modular Articulated Robotic Arm (MARA) environments. We support this process
using a framework that communicates with typical tools used in robotics, such
as Gazebo and Robot Operating System 2 (ROS 2). We evaluate several algorithms
in modular robots with an empirical study in simulation.
