  Unmanned Surface Vehicles technology (USVs) is an exciting topic that
essentially deploys an algorithm to safely and efficiently performs a mission.
Although reinforcement learning is a well-known approach to modeling such a
task, instability and divergence may occur when combining off-policy and
function approximation. In this work, we used deep reinforcement learning
combining Q-learning with a neural representation to avoid instability. Our
methodology uses deep q-learning and combines it with a rolling wave planning
approach on agile methodology. Our method contains two critical parts in order
to perform missions in an unknown environment. The first is a path planner that
is responsible for generating a potential effective path to a destination
without considering the details of the root. The latter is a decision-making
module that is responsible for short-term decisions on avoiding obstacles
during the near future steps of USV exploitation within the context of the
value function. Simulations were performed using two algorithms: a basic
vanilla vessel navigator (VVN) as a baseline and an improved one for the vessel
navigator with a planner and local view (VNPLV). Experimental results show that
the proposed method enhanced the performance of VVN by 55.31 on average for
long-distance missions. Our model successfully demonstrated obstacle avoidance
by means of deep reinforcement learning using planning adaptive paths in
unknown environments.
