  Human action Recognition for unknown views is a challenging task. We propose
a view-invariant deep human action recognition framework, which is a novel
integration of two important action cues: motion and shape temporal dynamics
(STD). The motion stream encapsulates the motion content of action as RGB
Dynamic Images (RGB-DIs) which are processed by the fine-tuned InceptionV3
model. The STD stream learns long-term view-invariant shape dynamics of action
using human pose model (HPM) based view-invariant features mined from
structural similarity index matrix (SSIM) based key depth human pose frames. To
predict the score of the test sample, three types of late fusion (maximum,
average and product) techniques are applied on individual stream scores. To
validate the performance of the proposed novel framework the experiments are
performed using both cross subject and cross-view validation schemes on three
publically available benchmarks- NUCLA multi-view dataset, UWA3D-II Activity
dataset and NTU RGB-D Activity dataset. Our algorithm outperforms with existing
state-of-the-arts significantly that is reported in terms of accuracy, receiver
operating characteristic (ROC) curve and area under the curve (AUC).
