  We introduce the first benchmark for a new problem --- recognizing human
action adverbs (HAA): "Adverbs Describing Human Actions" (ADHA). This is the
first step for computer vision to change over from pattern recognition to real
AI. We demonstrate some key features of ADHA: a semantically complete set of
adverbs describing human actions, a set of common, describable human actions,
and an exhaustive labeling of simultaneously emerging actions in each video. We
commit an in-depth analysis on the implementation of current effective models
in action recognition and image captioning on adverb recognition, and the
results show that such methods are unsatisfactory. Moreover, we propose a novel
three-stream hybrid model to deal the HAA problem, which achieves a better
result.
