  We introduce PlausiVL, a large video-language model for anticipating action
sequences that are plausible in the real-world. While significant efforts have
been made towards anticipating future actions, prior approaches do not take
into account the aspect of plausibility in an action sequence. To address this
limitation, we explore the generative capability of a large video-language
model in our work and further, develop the understanding of plausibility in an
action sequence by introducing two objective functions, a counterfactual-based
plausible action sequence learning loss and a long-horizon action repetition
loss. We utilize temporal logical constraints as well as verb-noun action pair
logical constraints to create implausible/counterfactual action sequences and
use them to train the model with plausible action sequence learning loss. This
loss helps the model to differentiate between plausible and not plausible
action sequences and also helps the model to learn implicit temporal cues
crucial for the task of action anticipation. The long-horizon action repetition
loss puts a higher penalty on the actions that are more prone to repetition
over a longer temporal window. With this penalization, the model is able to
generate diverse, plausible action sequences. We evaluate our approach on two
large-scale datasets, Ego4D and EPIC-Kitchens-100, and show improvements on the
task of action anticipation.
