  Generative models produce realistic objects in many domains, including text,
image, video, and audio synthesis. Most popular models---Generative Adversarial
Networks (GANs) and Variational Autoencoders (VAEs)---usually employ a standard
Gaussian distribution as a prior. Previous works show that the richer family of
prior distributions may help to avoid the mode collapse problem in GANs and to
improve the evidence lower bound in VAEs. We propose a new family of prior
distributions---Tensor Ring Induced Prior (TRIP)---that packs an exponential
number of Gaussians into a high-dimensional lattice with a relatively small
number of parameters. We show that these priors improve Fr\'echet Inception
Distance for GANs and Evidence Lower Bound for VAEs. We also study generative
models with TRIP in the conditional generation setup with missing conditions.
Altogether, we propose a novel plug-and-play framework for generative models
that can be utilized in any GAN and VAE-like architectures.
