  Feedback connections play a prominent role in the human brain but have not
received much attention in artificial neural network research. Here, a
biologically inspired feedback mechanism which gates rectified linear units is
proposed. On the MNIST dataset, autoencoders with feedback show faster
convergence, better performance, and more robustness to noise compared to their
counterparts without feedback. Some benefits, although less pronounced and less
consistent, can be observed when networks with feedback are applied on the
CIFAR-10 dataset.
