  We propose a novel autoencoding model called Pairwise Augmented GANs. We
train a generator and an encoder jointly and in an adversarial manner. The
generator network learns to sample realistic objects. In turn, the encoder
network at the same time is trained to map the true data distribution to the
prior in latent space. To ensure good reconstructions, we introduce an
augmented adversarial reconstruction loss. Here we train a discriminator to
distinguish two types of pairs: an object with its augmentation and the one
with its reconstruction. We show that such adversarial loss compares objects
based on the content rather than on the exact match. We experimentally
demonstrate that our model generates samples and reconstructions of quality
competitive with state-of-the-art on datasets MNIST, CIFAR10, CelebA and
achieves good quantitative results on CIFAR10.
