  Tabular data have been playing a mostly important role in diverse real-world
fields, such as healthcare, engineering, finance, etc. With the recent success
of deep learning, many tabular machine learning (ML) methods based on deep
networks (e.g., Transformer, ResNet) have achieved competitive performance on
tabular benchmarks. However, existing deep tabular ML methods suffer from the
representation entanglement and localization, which largely hinders their
prediction performance and leads to performance inconsistency on tabular tasks.
To overcome these problems, we explore a novel direction of applying prototype
learning for tabular ML and propose a prototype-based tabular representation
learning framework, PTaRL, for tabular prediction tasks. The core idea of PTaRL
is to construct prototype-based projection space (P-Space) and learn the
disentangled representation around global data prototypes. Specifically, PTaRL
mainly involves two stages: (i) Prototype Generation, that constructs global
prototypes as the basis vectors of P-Space for representation, and (ii)
Prototype Projection, that projects the data samples into P-Space and keeps the
core global data information via Optimal Transport. Then, to further acquire
the disentangled representations, we constrain PTaRL with two strategies: (i)
to diversify the coordinates towards global prototypes of different
representations within P-Space, we bring up a diversification constraint for
representation calibration; (ii) to avoid prototype entanglement in P-Space, we
introduce a matrix orthogonalization constraint to ensure the independence of
global prototypes. Finally, we conduct extensive experiments in PTaRL coupled
with state-of-the-art deep tabular ML models on various tabular benchmarks and
the results have shown our consistent superiority.
