  Learning long-horizon tasks such as navigation has presented difficult
challenges for successfully applying reinforcement learning to robotics. From
another perspective, under known environments, sampling-based planning can
robustly find collision-free paths in environments without learning. In this
work, we propose Control Transformer that models return-conditioned sequences
from low-level policies guided by a sampling-based Probabilistic Roadmap (PRM)
planner. We demonstrate that our framework can solve long-horizon navigation
tasks using only local information. We evaluate our approach on
partially-observed maze navigation with MuJoCo robots, including Ant, Point,
and Humanoid. We show that Control Transformer can successfully navigate
through mazes and transfer to unknown environments. Additionally, we apply our
method to a differential drive robot (Turtlebot3) and show zero-shot sim2real
transfer under noisy observations.
