  A new unified video analytics framework (ER3) is proposed for complex event
retrieval, recognition and recounting, based on the proposed video imprint
representation, which exploits temporal correlations among image features
across video frames. With the video imprint representation, it is convenient to
reverse map back to both temporal and spatial locations in video frames,
allowing for both key frame identification and key areas localization within
each frame. In the proposed framework, a dedicated feature alignment module is
incorporated for redundancy removal across frames to produce the tensor
representation, i.e., the video imprint. Subsequently, the video imprint is
individually fed into both a reasoning network and a feature aggregation
module, for event recognition/recounting and event retrieval tasks,
respectively. Thanks to its attention mechanism inspired by the memory networks
used in language modeling, the proposed reasoning network is capable of
simultaneous event category recognition and localization of the key pieces of
evidence for event recounting. In addition, the latent structure in our
reasoning network highlights the areas of the video imprint, which can be
directly used for event recounting. With the event retrieval task, the compact
video representation aggregated from the video imprint contributes to better
retrieval results than existing state-of-the-art methods.
