  In marine operations underwater manipulators play a primordial role. However,
due to uncertainties in the dynamic model and disturbances caused by the
environment, low-level control methods require great capabilities to adapt to
change. Furthermore, under position and torque constraints the requirements for
the control system are greatly increased. Reinforcement learning is a data
driven control technique that can learn complex control policies without the
need of a model. The learning capabilities of these type of agents allow for
great adaptability to changes in the operative conditions. In this article we
present a novel reinforcement learning low-level controller for the position
control of an underwater manipulator under torque and position constraints. The
reinforcement learning agent is based on an actor-critic architecture using
sensor readings as state information. Simulation results using the Reach Alpha
5 underwater manipulator show the advantages of the proposed control strategy.
