  Deep neural networks have demonstrated state-of-the-art performance in a
variety of real-world applications. In order to obtain performance gains, these
networks have grown larger and deeper, containing millions or even billions of
parameters and over a thousand layers. The trade-off is that these large
architectures require an enormous amount of memory, storage, and computation,
thus limiting their usability. Inspired by the recent tensor ring
factorization, we introduce Tensor Ring Networks (TR-Nets), which significantly
compress both the fully connected layers and the convolutional layers of deep
neural networks. Our results show that our TR-Nets approach {is able to
compress LeNet-5 by $11\times$ without losing accuracy}, and can compress the
state-of-the-art Wide ResNet by $243\times$ with only 2.3\% degradation in
{Cifar10 image classification}. Overall, this compression scheme shows promise
in scientific computing and deep learning, especially for emerging
resource-constrained devices such as smartphones, wearables, and IoT devices.
