  Split learning (SL) aims to protect user data privacy by distributing deep
models between client-server and keeping private data locally. Only processed
or `smashed' data can be transmitted from the clients to the server during the
SL process. However, recently proposed model inversion attacks can recover the
original data from the smashed data. In order to enhance privacy protection
against such attacks, a strategy is to adopt differential privacy (DP), which
involves safeguarding the smashed data at the expense of some accuracy loss.
This paper presents the first investigation into the impact on accuracy when
training multiple clients in SL with various privacy requirements.
Subsequently, we propose an approach that reviews the DP noise distributions of
other clients during client training to address the identified accuracy
degradation. We also examine the application of DP to the local model of SL to
gain insights into the trade-off between accuracy and privacy. Specifically,
findings reveal that introducing noise in the later local layers offers the
most favorable balance between accuracy and privacy. Drawing from our insights
in the shallower layers, we propose an approach to reduce the size of smashed
data to minimize data leakage while maintaining higher accuracy, optimizing the
accuracy-privacy trade-off. Additionally, a smaller size of smashed data
reduces communication overhead on the client side, mitigating one of the
notable drawbacks of SL. Experiments with popular datasets demonstrate that our
proposed approaches provide an optimal trade-off for incorporating DP into SL,
ultimately enhancing training accuracy for multi-client SL with varying privacy
requirements.
