  Contrastive learning is a model pre-training technique by first creating
similar views of the original data, and then encouraging the data and its
corresponding views to be close in the embedding space. Contrastive learning
has witnessed success in image and natural language data, thanks to the
domain-specific augmentation techniques that are both intuitive and effective.
Nonetheless, in tabular domain, the predominant augmentation technique for
creating views is through corrupting tabular entries via swapping values, which
is not as sound or effective. We propose a simple yet powerful improvement to
this augmentation technique: corrupting tabular data conditioned on class
identity. Specifically, when corrupting a specific tabular entry from an anchor
row, instead of randomly sampling a value in the same feature column from the
entire table uniformly, we only sample from rows that are identified to be
within the same class as the anchor row. We assume the semi-supervised learning
setting, and adopt the pseudo labeling technique for obtaining class identities
over all table rows. We also explore the novel idea of selecting features to be
corrupted based on feature correlation structures. Extensive experiments show
that the proposed approach consistently outperforms the conventional corruption
method for tabular data classification tasks. Our code is available at
https://github.com/willtop/Tabular-Class-Conditioned-SSL.
