  Context plays an important role in visual recognition. Recent studies have
shown that visual recognition networks can be fooled by placing objects in
inconsistent contexts (e.g., a cow in the ocean). To model the role of
contextual information in visual recognition, we systematically investigated
ten critical properties of where, when, and how context modulates recognition,
including the amount of context, context and object resolution, geometrical
structure of context, context congruence, and temporal dynamics of contextual
modulation. The tasks involved recognizing a target object surrounded with
context in a natural image. As an essential benchmark, we conducted a series of
psychophysics experiments where we altered one aspect of context at a time, and
quantified recognition accuracy. We propose a biologically-inspired
context-aware object recognition model consisting of a two-stream architecture.
The model processes visual information at the fovea and periphery in parallel,
dynamically incorporates object and contextual information, and sequentially
reasons about the class label for the target object. Across a wide range of
behavioral tasks, the model approximates human level performance without
retraining for each task, captures the dependence of context enhancement on
image properties, and provides initial steps towards integrating scene and
object information for visual recognition. All source code and data are
publicly available: https://github.com/kreimanlab/Put-In-Context.
