  Attention has become a common ingredient in deep learning architectures. It
adds a dynamical selection of information on top of the static selection of
information supported by weights. In the same way, we can imagine a
higher-order informational filter built on top of attention: an Attention
Schema (AS), namely, a descriptive and predictive model of attention. In
cognitive neuroscience, Attention Schema Theory (AST) supports this idea of
distinguishing attention from AS. A strong prediction of this theory is that an
agent can use its own AS to also infer the states of other agents' attention
and consequently enhance coordination with other agents. As such, multi-agent
reinforcement learning would be an ideal setting to experimentally test the
validity of AST. We explore different ways in which attention and AS interact
with each other. Our preliminary results indicate that agents that implement
the AS as a recurrent internal control achieve the best performance. In
general, these exploratory experiments suggest that equipping artificial agents
with a model of attention can enhance their social intelligence.
