  Since scenes are composed in part of objects, accurate recognition of scenes
requires knowledge about both scenes and objects. In this paper we address two
related problems: 1) scale induced dataset bias in multi-scale convolutional
neural network (CNN) architectures, and 2) how to combine effectively
scene-centric and object-centric knowledge (i.e. Places and ImageNet) in CNNs.
An earlier attempt, Hybrid-CNN, showed that incorporating ImageNet did not help
much. Here we propose an alternative method taking the scale into account,
resulting in significant recognition gains. By analyzing the response of
ImageNet-CNNs and Places-CNNs at different scales we find that both operate in
different scale ranges, so using the same network for all the scales induces
dataset bias resulting in limited performance. Thus, adapting the feature
extractor to each particular scale (i.e. scale-specific CNNs) is crucial to
improve recognition, since the objects in the scenes have their specific range
of scales. Experimental results show that the recognition accuracy highly
depends on the scale, and that simple yet carefully chosen multi-scale
combinations of ImageNet-CNNs and Places-CNNs, can push the state-of-the-art
recognition accuracy in SUN397 up to 66.26% (and even 70.17% with deeper
architectures, comparable to human performance).
