  Convolutional Neural Networks (CNNs) are successful deep learning models in
the field of computer vision. To get the maximum advantage of CNN model for
Human Action Recognition (HAR) using inertial sensor data, in this paper, we
use 4 types of spatial domain methods for transforming inertial sensor data to
activity images, which are then utilized in a novel fusion framework. These
four types of activity images are Signal Images (SI), Gramian Angular Field
(GAF) Images, Markov Transition Field (MTF) Images and Recurrence Plot (RP)
Images. Furthermore, for creating a multimodal fusion framework and to exploit
activity image, we made each type of activity images multimodal by convolving
with two spatial domain filters : Prewitt filter and High-boost filter.
Resnet-18, a CNN model, is used to learn deep features from multi-modalities.
Learned features are extracted from the last pooling layer of each ReNet and
then fused by canonical correlation based fusion (CCF) for improving the
accuracy of human action recognition. These highly informative features are
served as input to a multiclass Support Vector Machine (SVM). Experimental
results on three publicly available inertial datasets show the superiority of
the proposed method over the current state-of-the-art.
