  In this work, we propose an ensemble modeling approach for multimodal action
recognition. We independently train individual modality models using a variant
of focal loss tailored to handle the long-tailed distribution of the MECCANO
[21] dataset. Based on the underlying principle of focal loss, which captures
the relationship between tail (scarce) classes and their prediction
difficulties, we propose an exponentially decaying variant of focal loss for
our current task. It initially emphasizes learning from the hard misclassified
examples and gradually adapts to the entire range of examples in the dataset.
This annealing process encourages the model to strike a balance between
focusing on the sparse set of hard samples, while still leveraging the
information provided by the easier ones. Additionally, we opt for the late
fusion strategy to combine the resultant probability distributions from RGB and
Depth modalities for final action prediction. Experimental evaluations on the
MECCANO dataset demonstrate the effectiveness of our approach.
