  Although augmentations (e.g., perturbation of graph edges, image crops) boost
the efficiency of Contrastive Learning (CL), feature level augmentation is
another plausible, complementary yet not well researched strategy. Thus, we
present a novel spectral feature argumentation for contrastive learning on
graphs (and images). To this end, for each data view, we estimate a low-rank
approximation per feature map and subtract that approximation from the map to
obtain its complement. This is achieved by the proposed herein incomplete power
iteration, a non-standard power iteration regime which enjoys two valuable
byproducts (under mere one or two iterations): (i) it partially balances
spectrum of the feature map, and (ii) it injects the noise into rebalanced
singular values of the feature map (spectral augmentation). For two views, we
align these rebalanced feature maps as such an improved alignment step can
focus more on less dominant singular values of matrices of both views, whereas
the spectral augmentation does not affect the spectral angle alignment
(singular vectors are not perturbed). We derive the analytical form for: (i)
the incomplete power iteration to capture its spectrum-balancing effect, and
(ii) the variance of singular values augmented implicitly by the noise. We also
show that the spectral augmentation improves the generalization bound.
Experiments on graph/image datasets show that our spectral feature augmentation
outperforms baselines, and is complementary with other augmentation strategies
and compatible with various contrastive losses.
