  As neural networks become deeper, the redundancy within their parameters
increases. This phenomenon has led to several methods that attempt to reduce
the correlation between convolutional filters. We propose a computationally
efficient regularization technique that encourages orthonormality between
groups of filters within the same layer. Our experiments show that when
incorporated into recent adaptation methods for diffusion models and vision
transformers (ViTs), this regularization improves performance on downstream
tasks. We further show improved robustness when group orthogonality is enforced
during adversarial training. Our code is available at
https://github.com/YoavKurtz/GOR.
