  Previous works showed that Deep-RL can be applied to perform mapless
navigation, including the medium transition of Hybrid Unmanned Aerial
Underwater Vehicles (HUAUVs). This paper presents new approaches based on the
state-of-the-art actor-critic algorithms to address the navigation and medium
transition problems for a HUAUV. We show that a double critic Deep-RL with
Recurrent Neural Networks improves the navigation performance of HUAUVs using
solely range data and relative localization. Our Deep-RL approaches achieved
better navigation and transitioning capabilities with a solid generalization of
learning through distinct simulated scenarios, outperforming previous
approaches.
