  We propose a novel architecture design for video prediction in order to
utilize procedural domain knowledge directly as part of the computational graph
of data-driven models. On the basis of new challenging scenarios we show that
state-of-the-art video predictors struggle in complex dynamical settings, and
highlight that the introduction of prior process knowledge makes their learning
problem feasible. Our approach results in the learning of a symbolically
addressable interface between data-driven aspects in the model and our
dedicated procedural knowledge module, which we utilize in downstream control
tasks.
