  Researchers have applied deep neural networks to image restoration tasks, in
which they proposed various network architectures, loss functions, and training
methods. In particular, adversarial training, which is employed in recent
studies, seems to be a key ingredient to success. In this paper, we show that
simple convolutional autoencoders (CAEs) built upon only standard network
components, i.e., convolutional layers and skip connections, can outperform the
state-of-the-art methods which employ adversarial training and sophisticated
loss functions. The secret is to employ an evolutionary algorithm to
automatically search for good architectures. Training optimized CAEs by
minimizing the $\ell_2$ loss between reconstructed images and their ground
truths using the ADAM optimizer is all we need. Our experimental results show
that this approach achieves 27.8 dB peak signal to noise ratio (PSNR) on the
CelebA dataset and 40.4 dB on the SVHN dataset, compared to 22.8 dB and 33.0 dB
provided by the former state-of-the-art methods, respectively.
