  Video event extraction aims to detect salient events from a video and
identify the arguments for each event as well as their semantic roles. Existing
methods focus on capturing the overall visual scene of each frame, ignoring
fine-grained argument-level information. Inspired by the definition of events
as changes of states, we propose a novel framework to detect video events by
tracking the changes in the visual states of all involved arguments, which are
expected to provide the most informative evidence for the extraction of video
events. In order to capture the visual state changes of arguments, we decompose
them into changes in pixels within objects, displacements of objects, and
interactions among multiple arguments. We further propose Object State
Embedding, Object Motion-aware Embedding and Argument Interaction Embedding to
encode and track these changes respectively. Experiments on various video event
extraction tasks demonstrate significant improvements compared to
state-of-the-art models. In particular, on verb classification, we achieve
3.49% absolute gains (19.53% relative gains) in F1@5 on Video Situation
Recognition.
