  The central idea of contrastive learning is to discriminate between different
instances and force different views from the same instance to share the same
representation. To avoid trivial solutions, augmentation plays an important
role in generating different views, among which random cropping is shown to be
effective for the model to learn a generalized and robust representation.
Commonly used random crop operation keeps the distribution of the difference
between two views unchanged along the training process. In this work, we show
that adaptively controlling the disparity between two augmented views along the
training process enhances the quality of the learned representation.
Specifically, we present a parametric cubic cropping operation, ParamCrop, for
video contrastive learning, which automatically crops a 3D cubic by
differentiable 3D affine transformations. ParamCrop is trained simultaneously
with the video backbone using an adversarial objective and learns an optimal
cropping strategy from the data. The visualizations show that ParamCrop
adaptively controls the center distance and the IoU between two augmented
views, and the learned change in the disparity along the training process is
beneficial to learning a strong representation. Extensive ablation studies
demonstrate the effectiveness of the proposed ParamCrop on multiple contrastive
learning frameworks and video backbones. Codes and models will be available.
