  Many important tasks are defined in terms of object. To generalize across
these tasks, a reinforcement learning (RL) agent needs to exploit the structure
that the objects induce. Prior work has either hard-coded object-centric
features, used complex object-centric generative models, or updated state using
local spatial features. However, these approaches have had limited success in
enabling general RL agents. Motivated by this, we introduce "Feature-Attending
Recurrent Modules" (FARM), an architecture for learning state representations
that relies on simple, broadly applicable inductive biases for capturing
spatial and temporal regularities. FARM learns a state representation that is
distributed across multiple modules that each attend to spatiotemporal features
with an expressive feature attention mechanism. We show that this improves an
RL agent's ability to generalize across object-centric tasks. We study task
suites in both 2D and 3D environments and find that FARM better generalizes
compared to competing architectures that leverage attention or multiple
modules.
