  In many sequential decision-making tasks, the agent is not able to model the
full complexity of the world, which consists of multitudes of relevant and
irrelevant information. For example, a person walking along a city street who
tries to model all aspects of the world would quickly be overwhelmed by a
multitude of shops, cars, and people moving in and out of view, each following
their own complex and inscrutable dynamics. Is it possible to turn the agent's
firehose of sensory information into a minimal latent state that is both
necessary and sufficient for an agent to successfully act in the world? We
formulate this question concretely, and propose the Agent Control-Endogenous
State Discovery algorithm (AC-State), which has theoretical guarantees and is
practically demonstrated to discover the minimal control-endogenous latent
state which contains all of the information necessary for controlling the
agent, while fully discarding all irrelevant information. This algorithm
consists of a multi-step inverse model (predicting actions from distant
observations) with an information bottleneck. AC-State enables localization,
exploration, and navigation without reward or demonstrations. We demonstrate
the discovery of the control-endogenous latent state in three domains:
localizing a robot arm with distractions (e.g., changing lighting conditions
and background), exploring a maze alongside other agents, and navigating in the
Matterport house simulator.
