  Dense semantic forecasting anticipates future events in video by inferring
pixel-level semantics of an unobserved future image. We present a novel
approach that is applicable to various single-frame architectures and tasks.
Our approach consists of two modules. Feature-to-motion (F2M) module forecasts
a dense deformation field that warps past features into their future positions.
Feature-to-feature (F2F) module regresses the future features directly and is
therefore able to account for emergent scenery. The compound F2MF model
decouples the effects of motion from the effects of novelty in a task-agnostic
manner. We aim to apply F2MF forecasting to the most subsampled and the most
abstract representation of a desired single-frame model. Our design takes
advantage of deformable convolutions and spatial correlation coefficients
across neighbouring time instants. We perform experiments on three dense
prediction tasks: semantic segmentation, instance-level segmentation, and
panoptic segmentation. The results reveal state-of-the-art forecasting accuracy
across three dense prediction tasks.
