  This work focuses on the main challenges and problems in developing a virtual
oceanic environment reproducing real experiments using Unmanned Surface
Vehicles (USV) digital twins. We introduce the key features for building
virtual worlds, considering using Reinforcement Learning (RL) agents for
autonomous navigation and control. With this in mind, the main problems concern
the definition of the simulation equations (physics and mathematics), their
effective implementation, and how to include strategies for simulated control
and perception (sensors) to be used with RL. We present the modeling,
implementation steps, and challenges required to create a functional digital
twin based on a real robotic sailing vessel. The application is immediate for
developing navigation algorithms based on RL to be applied on real boats.
