  We present an end-to-end, model-based deep reinforcement learning agent which
dynamically attends to relevant parts of its state during planning. The agent
uses a bottleneck mechanism over a set-based representation to force the number
of entities to which the agent attends at each planning step to be small. In
experiments, we investigate the bottleneck mechanism with several sets of
customized environments featuring different challenges. We consistently observe
that the design allows the planning agents to generalize their learned
task-solving abilities in compatible unseen environments by attending to the
relevant objects, leading to better out-of-distribution generalization
performance.
