  When a person attempts to conceal an emotion, the genuine emotion is manifest
as a micro-expression. Exploration of automatic facial micro-expression
recognition systems is relatively new in the computer vision domain. This is
due to the difficulty in implementing optimal feature extraction methods to
cope with the subtlety and brief motion characteristics of the expression. Most
of the existing approaches extract the subtle facial movements based on
hand-crafted features. In this paper, we address the micro-expression
recognition task with a convolutional neural network (CNN) architecture, which
well integrates the features extracted from each video. A new feature
descriptor, Optical Flow Features from Apex frame Network (OFF-ApexNet) is
introduced. This feature descriptor combines the optical ow guided context with
the CNN. Firstly, we obtain the location of the apex frame from each video
sequence as it portrays the highest intensity of facial motion among all
frames. Then, the optical ow information are attained from the apex frame and a
reference frame (i.e., onset frame). Finally, the optical flow features are fed
into a pre-designed CNN model for further feature enhancement as well as to
carry out the expression classification. To evaluate the effectiveness of
OFF-ApexNet, comprehensive evaluations are conducted on three public
spontaneous micro-expression datasets (i.e., SMIC, CASME II and SAMM). The
promising recognition result suggests that the proposed method can optimally
describe the significant micro-expression details. In particular, we report
that, in a multi-database with leave-one-subject-out cross-validation
experimental protocol, the recognition performance reaches 74.60% of
recognition accuracy and F-measure of 71.04%. We also note that this is the
first work that performs cross-dataset validation on three databases in this
domain.
