  When applied to autonomous vehicle (AV) settings, action recognition can
enhance an environment model's situational awareness. This is especially
prevalent in scenarios where traditional geometric descriptions and heuristics
in AVs are insufficient. However, action recognition has traditionally been
studied for humans, and its limited adaptability to noisy, un-clipped,
un-pampered, raw RGB data has limited its application in other fields. To push
for the advancement and adoption of action recognition into AVs, this work
proposes a novel two-stage action recognition system, termed RALACs. RALACs
formulates the problem of action recognition for road scenes, and bridges the
gap between it and the established field of human action recognition. This work
shows how attention layers can be useful for encoding the relations across
agents, and stresses how such a scheme can be class-agnostic. Furthermore, to
address the dynamic nature of agents on the road, RALACs constructs a novel
approach to adapting Region of Interest (ROI) Alignment to agent tracks for
downstream action classification. Finally, our scheme also considers the
problem of active agent detection, and utilizes a novel application of fusing
optical flow maps to discern relevant agents in a road scene. We show that our
proposed scheme can outperform the baseline on the ICCV2021 Road Challenge
dataset and by deploying it on a real vehicle platform, we provide preliminary
insight to the usefulness of action recognition in decision making.
