  Object-based attention is a key component of the visual system, relevant for
perception, learning, and memory. Neurons tuned to features of attended objects
tend to be more active than those associated with non-attended objects. There
is a rich set of models of this phenomenon in computational neuroscience.
However, there is currently a divide between models that successfully match
physiological data but can only deal with extremely simple problems and models
of attention used in computer vision. For example, attention in the brain is
known to depend on top-down processing, whereas self-attention in deep learning
does not. Here, we propose an artificial neural network model of object-based
attention that captures the way in which attention is both top-down and
recurrent. Our attention model works well both on simple test stimuli, such as
those using images of handwritten digits, and on more complex stimuli, such as
natural images drawn from the COCO dataset. We find that our model replicates a
range of findings from neuroscience, including attention-invariant tuning,
inhibition of return, and attention-mediated scaling of activity. Understanding
object based attention is both computationally interesting and a key problem
for computational neuroscience.
