  Privacy issues were raised in the process of training deep learning in
medical, mobility, and other fields. To solve this problem, we present
privacy-preserving distributed deep learning method that allow clients to learn
a variety of data without direct exposure. We divided a single deep learning
architecture into a common extractor, a cloud model and a local classifier for
the distributed learning. First, the common extractor, which is used by local
clients, extracts secure features from the input data. The secure features also
take the role that the cloud model can employ various task and diverse types of
data. The feature contain the most important information that helps to proceed
various task. Second, the cloud model including most parts of the whole
training model gets the embedded features from the massive local clients, and
performs most of deep learning operations which takes severe computing cost.
After the operations in cloud model finished, outputs of the cloud model send
back to local clients. Finally, the local classifier determined classification
results and delivers the results to local clients. When clients train models,
our model does not directly expose sensitive information to exterior network.
During the test, the average performance improvement was 2.63% over the
existing local training model. However, in a distributed environment, there is
a possibility of inversion attack due to exposed features. For this reason, we
experimented with the common extractor to prevent data restoration. The quality
of restoration of the original image was tested by adjusting the depth of the
common extractor. As a result, we found that the deeper the common extractor,
the restoration score decreased to 89.74.
