  Dynamic patterns are characterized by complex spatial and motion patterns.
Understanding dynamic patterns requires a disentangled representational model
that separates the factorial components. A commonly used model for dynamic
patterns is the state space model, where the state evolves over time according
to a transition model and the state generates the observed image frames
according to an emission model. To model the motions explicitly, it is natural
for the model to be based on the motions or the displacement fields of the
pixels. Thus in the emission model, we let the hidden state generate the
displacement field, which warps the trackable component in the previous image
frame to generate the next frame while adding a simultaneously emitted residual
image to account for the change that cannot be explained by the deformation.
The warping of the previous image is about the trackable part of the change of
image frame, while the residual image is about the intrackable part of the
image. We use a maximum likelihood algorithm to learn the model that iterates
between inferring latent noise vectors that drive the transition model and
updating the parameters given the inferred latent vectors. Meanwhile we adopt a
regularization term to penalize the norms of the residual images to encourage
the model to explain the change of image frames by trackable motion. Unlike
existing methods on dynamic patterns, we learn our model in unsupervised
setting without ground truth displacement fields. In addition, our model
defines a notion of intrackability by the separation of warped component and
residual component in each image frame. We show that our method can synthesize
realistic dynamic pattern, and disentangling appearance, trackable and
intrackable motions. The learned models are useful for motion transfer, and it
is natural to adopt it to define and measure intrackability of a dynamic
pattern.
