  In this paper, we present an exploration and assessment of employing a
centralized deep Q-network (DQN) controller as a substitute for the prevalent
use of PID controllers in the context of 6DOF swimming robots. Our primary
focus centers on illustrating this transition with the specific case of
underwater object tracking. DQN offers advantages such as data efficiency and
off-policy learning, while remaining simpler to implement than other
reinforcement learning methods. Given the absence of a dynamic model for our
robot, we propose an RL agent to control this multi-input-multi-output (MIMO)
system, where a centralized controller may offer more robust control than
distinct PIDs. Our approach involves initially using classical controllers for
safe exploration, then gradually shifting to DQN to take full control of the
robot.
  We divide the underwater tracking task into vision and control modules. We
use established methods for vision-based tracking and introduce a centralized
DQN controller. By transmitting bounding box data from the vision module to the
control module, we enable adaptation to various objects and effortless vision
system replacement. Furthermore, dealing with low-dimensional data facilitates
cost-effective online learning for the controller. Our experiments, conducted
within a Unity-based simulator, validate the effectiveness of a centralized RL
agent over separated PID controllers, showcasing the applicability of our
framework for training the underwater RL agent and improved performance
compared to traditional control methods. The code for both real and simulation
implementations is at https://github.com/FARAZLOTFI/underwater-object-tracking.
