  We investigate the scenario that a robot needs to reach a designated goal
after taking a sequence of appropriate actions in a non-static environment that
is partially structured. One application example is to control a marine vehicle
to move in the ocean. The ocean environment is dynamic and oftentimes the ocean
waves result in strong disturbances that can disturb the vehicle's motion.
Modeling such dynamic environment is non-trivial, and integrating such model in
the robotic motion control is particularly difficult. Fortunately, the ocean
currents usually form some local patterns (e.g. vortex) and thus the
environment is partially structured. The historically observed data can be used
to train the robot to learn to interact with the ocean tidal disturbances. In
this paper we propose a method that applies the deep reinforcement learning
framework to learn such partially structured complex disturbances. Our results
show that, by training the robot under artificial and real ocean disturbances,
the robot is able to successfully act in complex and spatiotemporal
environments.
