  Leveraging physical knowledge described by partial differential equations
(PDEs) is an appealing way to improve unsupervised video prediction methods.
Since physics is too restrictive for describing the full visual content of
generic videos, we introduce PhyDNet, a two-branch deep architecture, which
explicitly disentangles PDE dynamics from unknown complementary information. A
second contribution is to propose a new recurrent physical cell (PhyCell),
inspired from data assimilation techniques, for performing PDE-constrained
prediction in latent space. Extensive experiments conducted on four various
datasets show the ability of PhyDNet to outperform state-of-the-art methods.
Ablation studies also highlight the important gain brought out by both
disentanglement and PDE-constrained prediction. Finally, we show that PhyDNet
presents interesting features for dealing with missing data and long-term
forecasting.
