  As a practical privacy-preserving learning method, split learning has drawn
much attention in academia and industry. However, its security is constantly
being questioned since the intermediate results are shared during training and
inference. In this paper, we focus on the privacy leakage from the forward
embeddings of split learning. Specifically, since the forward embeddings
contain too much information about the label, the attacker can either use a few
labeled samples to fine-tune the top model or perform unsupervised attacks such
as clustering to infer the true labels from the forward embeddings. To prevent
such kind of privacy leakage, we propose the potential energy loss to make the
forward embeddings become more 'complicated', by pushing embeddings of the same
class towards the decision boundary. Therefore, it is hard for the attacker to
learn from the forward embeddings. Experiment results show that our method
significantly lowers the performance of both fine-tuning attacks and clustering
attacks.
