  Convolutional neural network (CNN) models have achieved great success in many
fields. With the advent of ResNet, networks used in practice are getting deeper
and wider. However, is each layer non-trivial in networks? To answer this
question, we trained a network on the training set, then we replace the network
convolution kernels with zeros and test the result models on the test set. We
compared experimental results with baseline and showed that we can reach
similar or even the same performances. Although convolution kernels are the
cores of networks, we demonstrate that some of them are trivial and regular in
ResNet.
