  Managing novelty in perception-based human activity recognition (HAR) is
critical in realistic settings to improve task performance over time and ensure
solution generalization outside of prior seen samples. Novelty manifests in HAR
as unseen samples, activities, objects, environments, and sensor changes, among
other ways. Novelty may be task-relevant, such as a new class or new features,
or task-irrelevant resulting in nuisance novelty, such as never before seen
noise, blur, or distorted video recordings. To perform HAR optimally,
algorithmic solutions must be tolerant to nuisance novelty, and learn over time
in the face of novelty. This paper 1) formalizes the definition of novelty in
HAR building upon the prior definition of novelty in classification tasks, 2)
proposes an incremental open world learning (OWL) protocol and applies it to
the Kinetics datasets to generate a new benchmark KOWL-718, 3) analyzes the
performance of current state-of-the-art HAR models when novelty is introduced
over time, 4) provides a containerized and packaged pipeline for reproducing
the OWL protocol and for modifying for any future updates to Kinetics. The
experimental analysis includes an ablation study of how the different models
perform under various conditions as annotated by Kinetics-AVA. The protocol as
an algorithm for reproducing experiments using the KOWL-718 benchmark will be
publicly released with code and containers at
https://github.com/prijatelj/human-activity-recognition-in-an-open-world. The
code may be used to analyze different annotations and subsets of the Kinetics
datasets in an incremental open world fashion, as well as be extended as
further updates to Kinetics are released.
