  Foundation models (FMs) are large neural networks trained on broad datasets,
excelling in downstream tasks with minimal fine-tuning. Human activity
recognition in video has advanced with FMs, driven by competition among
different architectures. However, high accuracies on standard benchmarks can
draw an artificially rosy picture, as they often overlook real-world factors
like changing camera perspectives. Popular benchmarks, mostly from YouTube or
movies, offer diverse views but only coarse actions, which are insufficient for
use-cases needing fine-grained, domain-specific actions. Domain-specific
datasets (e.g., for industrial assembly) typically use data from limited static
perspectives. This paper empirically evaluates how perspective changes affect
different FMs in fine-grained human activity recognition. We compare multiple
backbone architectures and design choices, including image- and video- based
models, and various strategies for temporal information fusion, including
commonly used score averaging and more novel attention-based temporal
aggregation mechanisms. This is the first systematic study of different
foundation models and specific design choices for human activity recognition
from unknown views, conducted with the goal to provide guidance for backbone-
and temporal- fusion scheme selection. Code and models will be made publicly
available to the community.
