  We propose LESS-VFL, a communication-efficient feature selection method for
distributed systems with vertically partitioned data. We consider a system of a
server and several parties with local datasets that share a sample ID space but
have different feature sets. The parties wish to collaboratively train a model
for a prediction task. As part of the training, the parties wish to remove
unimportant features in the system to improve generalization, efficiency, and
explainability. In LESS-VFL, after a short pre-training period, the server
optimizes its part of the global model to determine the relevant outputs from
party models. This information is shared with the parties to then allow local
feature selection without communication. We analytically prove that LESS-VFL
removes spurious features from model training. We provide extensive empirical
evidence that LESS-VFL can achieve high accuracy and remove spurious features
at a fraction of the communication cost of other feature selection approaches.
