  We propose SplitNet, a method for decoupling visual perception and policy
learning. By incorporating auxiliary tasks and selective learning of portions
of the model, we explicitly decompose the learning objectives for visual
navigation into perceiving the world and acting on that perception. We show
dramatic improvements over baseline models on transferring between simulators,
an encouraging step towards Sim2Real. Additionally, SplitNet generalizes better
to unseen environments from the same simulator and transfers faster and more
effectively to novel embodied navigation tasks. Further, given only a small
sample from a target domain, SplitNet can match the performance of traditional
end-to-end pipelines which receive the entire dataset. Code is available
https://github.com/facebookresearch/splitnet
