  A key capability of an intelligent system is deciding when events from past
experience must be remembered and when they can be forgotten. Towards this
goal, we develop a predictive model of human visual event memory and how those
memories decay over time. We introduce Memento10k, a new, dynamic video
memorability dataset containing human annotations at different viewing delays.
Based on our findings we propose a new mathematical formulation of memorability
decay, resulting in a model that is able to produce the first quantitative
estimation of how a video decays in memory over time. In contrast with previous
work, our model can predict the probability that a video will be remembered at
an arbitrary delay. Importantly, our approach combines visual and semantic
information (in the form of textual captions) to fully represent the meaning of
events. Our experiments on two video memorability benchmarks, including
Memento10k, show that our model significantly improves upon the best prior
approach (by 12% on average).
