  Many aesthetic models in computer vision suffer from two shortcomings: 1) the
low descriptiveness and interpretability of those hand-crafted aesthetic
criteria (i.e., nonindicative of region-level aesthetics), and 2) the
difficulty of engineering aesthetic features adaptively and automatically
toward different image sets. To remedy these problems, we develop a deep
architecture to learn aesthetically-relevant visual attributes from Flickr1,
which are localized by multiple textual attributes in a weakly-supervised
setting. More specifically, using a bag-ofwords (BoW) representation of the
frequent Flickr image tags, a sparsity-constrained subspace algorithm discovers
a compact set of textual attributes (e.g., landscape and sunset) for each
image. Then, a weakly-supervised learning algorithm projects the textual
attributes at image-level to the highly-responsive image patches at
pixel-level. These patches indicate where humans look at appealing regions with
respect to each textual attribute, which are employed to learn the visual
attributes. Psychological and anatomical studies have shown that humans
perceive visual concepts hierarchically. Hence, we normalize these patches and
feed them into a five-layer convolutional neural network (CNN) to mimick the
hierarchy of human perceiving the visual attributes. We apply the learned deep
features on image retargeting, aesthetics ranking, and retrieval. Both
subjective and objective experimental results thoroughly demonstrate the
competitiveness of our approach.
