  Action classification in still images has been a popular research topic in
computer vision. Labelling large scale datasets for action classification
requires tremendous manual work, which is hard to scale up. Besides, the action
categories in such datasets are pre-defined and vocabularies are fixed. However
humans may describe the same action with different phrases, which leads to the
difficulty of vocabulary expansion for traditional fully-supervised methods. We
observe that large amounts of images with sentence descriptions are readily
available on the Internet. The sentence descriptions can be regarded as weak
labels for the images, which contain rich information and could be used to
learn flexible expressions of action categories. We propose a method to learn
an Action Concept Tree (ACT) and an Action Semantic Alignment (ASA) model for
classification from image-description data via a two-stage learning process. A
new dataset for the task of learning actions from descriptions is built.
Experimental results show that our method outperforms several baseline methods
significantly.
