  Anticipating future events is an essential feature for intelligent systems
and embodied AI. However, compared to the traditional recognition task, the
uncertainty of future and reasoning ability requirement make the anticipation
task very challenging and far beyond solved. In this filed, previous methods
usually care more about the model architecture design or but few attention has
been put on how to train an anticipation model with a proper learning policy.
To this end, in this work, we propose a novel training scheme called Dynamic
Context Removal (DCR), which dynamically schedules the visibility of observed
future in the learning procedure. It follows the human-like curriculum learning
process, i.e., gradually removing the event context to increase the
anticipation difficulty till satisfying the final anticipation target. Our
learning scheme is plug-and-play and easy to integrate any reasoning model
including transformer and LSTM, with advantages in both effectiveness and
efficiency. In extensive experiments, the proposed method achieves
state-of-the-art on four widely-used benchmarks. Our code and models are
publicly released at https://github.com/AllenXuuu/DCR.
