  Recently, infrared human action recognition has attracted increasing
attention for it has many advantages over visible light, that is, being robust
to illumination change and shadows. However, the infrared action data is
limited until now, which degrades the performance of infrared action
recognition. Motivated by the idea of transfer learning, an infrared human
action recognition framework using auxiliary data from visible light is
proposed to solve the problem of limited infrared action data. In the proposed
framework, we first construct a novel Cross-Dataset Feature Alignment and
Generalization (CDFAG) framework to map the infrared data and visible light
data into a common feature space, where Kernel Manifold Alignment (KEMA) and a
dual alignedto-generalized encoders (AGE) model are employed to represent the
feature. Then, a support vector machine (SVM) is trained, using both the
infrared data and visible light data, and can classify the features derived
from infrared data. The proposed method is evaluated on InfAR, which is a
publicly available infrared human action dataset. To build up auxiliary data,
we set up a novel visible light action dataset XD145. Experimental results show
that the proposed method can achieve state-of-the-art performance compared with
several transfer learning and domain adaptation methods.
