  We develop a model for representing visual texture in a low-dimensional
feature space, along with a novel self-supervised learning objective that is
used to train it on an unlabeled database of texture images. Inspired by the
architecture of primate visual cortex, the model uses a first stage of oriented
linear filters (corresponding to cortical area V1), consisting of both
rectified units (simple cells) and pooled phase-invariant units (complex
cells). These responses are processed by a second stage (analogous to cortical
area V2) consisting of convolutional filters followed by half-wave
rectification and pooling to generate V2 'complex cell' responses. The second
stage filters are trained on a set of unlabeled homogeneous texture images,
using a novel contrastive objective that maximizes the distance between the
distribution of V2 responses to individual images and the distribution of
responses across all images. When evaluated on texture classification, the
trained model achieves substantially greater data-efficiency than a variety of
deep hierarchical model architectures. Moreover, we show that the learned model
exhibits stronger representational similarity to texture responses of neural
populations recorded in primate V2 than pre-trained deep CNNs.
