  Object goal navigation aims to navigate an agent to locations of a given
object category in unseen environments. Classical methods explicitly build maps
of environments and require extensive engineering while lacking semantic
information for object-oriented exploration. On the other hand, end-to-end
learning methods alleviate manual map design and predict actions using implicit
representations. Such methods, however, lack an explicit notion of geometry and
may have limited ability to encode navigation history. In this work, we propose
an implicit spatial map for object goal navigation. Our implicit map is
recursively updated with new observations at each step using a transformer. To
encourage spatial reasoning, we introduce auxiliary tasks and train our model
to reconstruct explicit maps as well as to predict visual features, semantic
labels and actions. Our method significantly outperforms the state of the art
on the challenging MP3D dataset and generalizes well to the HM3D dataset. We
successfully deploy our model on a real robot and achieve encouraging object
goal navigation results in real scenes using only a few real-world
demonstrations. Code, trained models and videos are available at
\url{https://www.di.ens.fr/willow/research/onav_rim/}.
