  Human beings often assess the aesthetic quality of an image coupled with the
identification of the image's semantic content. This paper addresses the
correlation issue between automatic aesthetic quality assessment and semantic
recognition. We cast the assessment problem as the main task among a multi-task
deep model, and argue that semantic recognition task offers the key to address
this problem. Based on convolutional neural networks, we employ a single and
simple multi-task framework to efficiently utilize the supervision of aesthetic
and semantic labels. A correlation item between these two tasks is further
introduced to the framework by incorporating the inter-task relationship
learning. This item not only provides some useful insight about the correlation
but also improves assessment accuracy of the aesthetic task. Particularly, an
effective strategy is developed to keep a balance between the two tasks, which
facilitates to optimize the parameters of the framework. Extensive experiments
on the challenging AVA dataset and Photo.net dataset validate the importance of
semantic recognition in aesthetic quality assessment, and demonstrate that
multi-task deep models can discover an effective aesthetic representation to
achieve state-of-the-art results.
