  In this work, we present a novel Reinforcement Learning (RL) algorithm for
the off-road trajectory tracking problem. Off-road environments involve varying
terrain types and elevations, and it is difficult to model the interaction
dynamics of specific off-road vehicles with such a diverse and complex
environment. Standard RL policies trained on a simulator will fail to operate
in such challenging real-world settings. Instead of using a naive domain
randomization approach, we propose an innovative supervised-learning based
approach for overcoming the sim-to-real gap problem. Our approach efficiently
exploits the limited real-world data available to adapt the baseline RL policy
obtained using a simple kinematics simulator. This avoids the need for modeling
the diverse and complex interaction of the vehicle with off-road environments.
We evaluate the performance of the proposed algorithm using two different
off-road vehicles, Warthog and Moose. Compared to the standard ILQR approach,
our proposed approach achieves a 30% and 50% reduction in cross track error in
Warthog and Moose, respectively, by utilizing only 30 minutes of real-world
driving data.
