  We address planning and navigation in challenging 3D video games featuring
maps with disconnected regions reachable by agents using special actions. In
this setting, classical symbolic planners are not applicable or difficult to
adapt. We introduce a hybrid technique combining a low level policy trained
with reinforcement learning and a graph based high level classical planner. In
addition to providing human-interpretable paths, the approach improves the
generalization performance of an end-to-end approach in unseen maps, where it
achieves a 20% absolute increase in success rate over a recurrent end-to-end
agent on a point to point navigation task in yet unseen large-scale maps of
size 1km x 1km. In an in-depth experimental study, we quantify the limitations
of end-to-end Deep RL approaches in vast environments and we also introduce
"GameRLand3D", a new benchmark and soon to be released environment can generate
complex procedural 3D maps for navigation tasks.
