  How do we know that a kitchen is a kitchen by looking? Relatively little is
known about how we conceptualize and categorize different visual environments.
Traditional models of visual perception posit that scene categorization is
achieved through the recognition of a scene's objects, yet these models cannot
account for the mounting evidence that human observers are relatively
insensitive to the local details in an image. Psychologists have long theorized
that the affordances, or actionable possibilities of a stimulus are pivotal to
its perception. To what extent are scene categories created from similar
affordances? Using a large-scale experiment using hundreds of scene categories,
we show that the activities afforded by a visual scene provide a fundamental
categorization principle. Affordance-based similarity explained the majority of
the structure in the human scene categorization patterns, outperforming
alternative similarities based on objects or visual features. We all models
were combined, affordances provided the majority of the predictive power in the
combined model, and nearly half of the total explained variance is captured
only by affordances. These results challenge many existing models of high-level
visual perception, and provide immediately testable hypotheses for the
functional organization of the human perceptual system.
