  Emotion is an essential part of Artificial Intelligence (AI) and human mental
health. Current emotion recognition research mainly focuses on single modality
(e.g., facial expression), while human emotion expressions are multi-modal in
nature. In this paper, we propose a hybrid emotion recognition system
leveraging two emotion-rich and tightly-coupled modalities, i.e., facial
expression and body gesture. However, unbiased and fine-grained facial
expression and gesture recognition remain a major problem. To this end, unlike
our rivals relying on contact or even invasive sensors, we explore the
commodity WiFi signal for device-free and contactless gesture recognition,
while adopting a vision-based facial expression. However, there exist two
design challenges, i.e., how to improve the sensitivity of WiFi signals and how
to process the large-volume, heterogeneous, and non-synchronous data
contributed by the two-modalities. For the former, we propose a signal
sensitivity enhancement method based on the Rician K factor theory; for the
latter, we combine CNN and RNN to mine the high-level features of bi-modal
data, and perform a score-level fusion for fine-grained recognition. To
evaluate the proposed method, we build a first-of-its-kind Vision-CSI Emotion
Database (VCED) and conduct extensive experiments. Empirical results show the
superiority of the bi-modality by achieving 83.24\% recognition accuracy for
seven emotions, as compared with 66.48% and 66.67% recognition accuracy by
gesture-only based solution and facial-only based solution, respectively. The
VCED database download link is https://github.com/purpleleaves007/WIFE-Dataset.
