  Recent work in Video Frame Interpolation (VFI) tries to formulate VFI as a
diffusion-based conditional image generation problem, synthesizing the
intermediate frame given a random noise and neighboring frames. Due to the
relatively high resolution of videos, Latent Diffusion Models (LDMs) are
employed as the conditional generation model, where the autoencoder compresses
images into latent representations for diffusion and then reconstructs images
from these latent representations. Such a formulation poses a crucial
challenge: VFI expects that the output is deterministically equal to the ground
truth intermediate frame, but LDMs randomly generate a diverse set of different
images when the model runs multiple times. The reason for the diverse
generation is that the cumulative variance (variance accumulated at each step
of generation) of generated latent representations in LDMs is large. This makes
the sampling trajectory random, resulting in diverse rather than deterministic
generations. To address this problem, we propose our unique solution: Frame
Interpolation with Consecutive Brownian Bridge Diffusion. Specifically, we
propose consecutive Brownian Bridge diffusion that takes a deterministic
initial value as input, resulting in a much smaller cumulative variance of
generated latent representations. Our experiments suggest that our method can
improve together with the improvement of the autoencoder and achieve
state-of-the-art performance in VFI, leaving strong potential for further
enhancement.
