  Tensor decompositions have been successfully applied to compress neural
networks. The compression algorithms using tensor decompositions commonly
minimize the approximation error on the weights. Recent work assumes the
approximation error on the weights is a proxy for the performance of the model
to compress multiple layers and fine-tune the compressed model. Surprisingly,
little research has systematically evaluated which approximation errors can be
used to make choices regarding the layer, tensor decomposition method, and
level of compression. To close this gap, we perform an experimental study to
test if this assumption holds across different layers and types of
decompositions, and what the effect of fine-tuning is. We include the
approximation error on the features resulting from a compressed layer in our
analysis to test if this provides a better proxy, as it explicitly takes the
data into account. We find the approximation error on the weights has a
positive correlation with the performance error, before as well as after
fine-tuning. Basing the approximation error on the features does not improve
the correlation significantly. While scaling the approximation error commonly
is used to account for the different sizes of layers, the average correlation
across layers is smaller than across all choices (i.e. layers, decompositions,
and level of compression) before fine-tuning. When calculating the correlation
across the different decompositions, the average rank correlation is larger
than across all choices. This means multiple decompositions can be considered
for compression and the approximation error can be used to choose between them.
