  Existing methods for open-set action recognition focus on novelty detection
that assumes video clips show a single action, which is unrealistic in the real
world. We propose a new method for open set action recognition and novelty
detection via MUlti-Label Evidential learning (MULE), that goes beyond previous
novel action detection methods by addressing the more general problems of
single or multiple actors in the same scene, with simultaneous action(s) by any
actor. Our Beta Evidential Neural Network estimates multi-action uncertainty
with Beta densities based on actor-context-object relation representations. An
evidence debiasing constraint is added to the objective function for
optimization to reduce the static bias of video representations, which can
incorrectly correlate predictions and static cues. We develop a learning
algorithm based on a primal-dual average scheme update to optimize the proposed
problem. Theoretical analysis of the optimization algorithm demonstrates the
convergence of the primal solution sequence and bounds for both the loss
function and the debiasing constraint. Uncertainty and belief-based novelty
estimation mechanisms are formulated to detect novel actions. Extensive
experiments on two real-world video datasets show that our proposed approach
achieves promising performance in single/multi-actor, single/multi-action
settings.
