  In this work, we propose a Model Predictive Control (MPC)-based Reinforcement
Learning (RL) method for Autonomous Surface Vehicles (ASVs). The objective is
to find an optimal policy that minimizes the closed-loop performance of a
simplified freight mission, including collision-free path following, autonomous
docking, and a skillful transition between them. We use a parametrized
MPC-scheme to approximate the optimal policy, which considers
path-following/docking costs and states (position, velocity)/inputs (thruster
force, angle) constraints. The Least Squares Temporal Difference (LSTD)-based
Deterministic Policy Gradient (DPG) method is then applied to update the policy
parameters. Our simulation results demonstrate that the proposed MPC-LSTD-based
DPG method could improve the closed-loop performance during learning for the
freight mission problem of ASV.
