  The ability for an agent to localize itself within an environment is crucial
for many real-world applications. For unknown environments, Simultaneous
Localization and Mapping (SLAM) enables incremental and concurrent building of
and localizing within a map. We present a new, differentiable architecture,
Neural Graph Optimizer, progressing towards a complete neural network solution
for SLAM by designing a system composed of a local pose estimation model, a
novel pose selection module, and a novel graph optimization process. The entire
architecture is trained in an end-to-end fashion, enabling the network to
automatically learn domain-specific features relevant to the visual odometry
and avoid the involved process of feature engineering. We demonstrate the
effectiveness of our system on a simulated 2D maze and the 3D ViZ-Doom
environment.
