  We present a learning algorithm for human activity recognition in videos. Our
approach is designed for UAV videos, which are mainly acquired from obliquely
placed dynamic cameras that contain a human actor along with background motion.
Typically, the human actors occupy less than one-tenth of the spatial
resolution. Our approach simultaneously harnesses the benefits of frequency
domain representations, a classical analysis tool in signal processing, and
data driven neural networks. We build a differentiable static-dynamic frequency
mask prior to model the salient static and dynamic pixels in the video, crucial
for the underlying task of action recognition. We use this differentiable mask
prior to enable the neural network to intrinsically learn disentangled feature
representations via an identity loss function. Our formulation empowers the
network to inherently compute disentangled salient features within its layers.
Further, we propose a cost-function encapsulating temporal relevance and
spatial content to sample the most important frame within uniformly spaced
video segments. We conduct extensive experiments on the UAV Human dataset and
the NEC Drone dataset and demonstrate relative improvements of 5.72% - 13.00%
over the state-of-the-art and 14.28% - 38.05% over the corresponding baseline
model.
