  Generating videos is a complex task that is accomplished by generating a set
of temporally coherent images frame-by-frame. This limits the expressivity of
videos to only image-based operations on the individual video frames needing
network designs to obtain temporally coherent trajectories in the underlying
image space. We propose INR-V, a video representation network that learns a
continuous space for video-based generative tasks. INR-V parameterizes videos
using implicit neural representations (INRs), a multi-layered perceptron that
predicts an RGB value for each input pixel location of the video. The INR is
predicted using a meta-network which is a hypernetwork trained on neural
representations of multiple video instances. Later, the meta-network can be
sampled to generate diverse novel videos enabling many downstream video-based
generative tasks. Interestingly, we find that conditional regularization and
progressive weight initialization play a crucial role in obtaining INR-V. The
representation space learned by INR-V is more expressive than an image space
showcasing many interesting properties not possible with the existing works.
For instance, INR-V can smoothly interpolate intermediate videos between known
video instances (such as intermediate identities, expressions, and poses in
face videos). It can also in-paint missing portions in videos to recover
temporally coherent full videos. In this work, we evaluate the space learned by
INR-V on diverse generative tasks such as video interpolation, novel video
generation, video inversion, and video inpainting against the existing
baselines. INR-V significantly outperforms the baselines on several of these
demonstrated tasks, clearly showcasing the potential of the proposed
representation space.
