  In this paper, we propose a convolutional neural network(CNN) with 3-D rank-1
filters which are composed by the outer product of 1-D filters. After being
trained, the 3-D rank-1 filters can be decomposed into 1-D filters in the test
time for fast inference. The reason that we train 3-D rank-1 filters in the
training stage instead of consecutive 1-D filters is that a better gradient
flow can be obtained with this setting, which makes the training possible even
in the case where the network with consecutive 1-D filters cannot be trained.
The 3-D rank-1 filters are updated by both the gradient flow and the outer
product of the 1-D filters in every epoch, where the gradient flow tries to
obtain a solution which minimizes the loss function, while the outer product
operation tries to make the parameters of the filter to live on a rank-1
sub-space. Furthermore, we show that the convolution with the rank-1 filters
results in low rank outputs, constraining the final output of the CNN also to
live on a low dimensional subspace.
