  Low-rank approximation is an effective model compression technique to not
only reduce parameter storage requirements, but to also reduce computations.
For convolutional neural networks (CNNs), however, well-known low-rank
approximation methods, such as Tucker or CP decomposition, result in degraded
model accuracy because decomposed layers hinder training convergence. In this
paper, we propose a new training technique that finds a flat minimum in the
view of low-rank approximation without a decomposed structure during training.
By preserving the original model structure, 2-dimensional low-rank
approximation demanding lowering (such as im2col) is available in our proposed
scheme. We show that CNN models can be compressed by low-rank approximation
with much higher compression ratio than conventional training methods while
maintaining or even enhancing model accuracy. We also discuss various
2-dimensional low-rank approximation techniques for CNNs.
