  The performance of machine learning algorithms heavily relies on the
availability of a large amount of training data. However, in reality, data
usually reside in distributed parties such as different institutions and may
not be directly gathered and integrated due to various data policy constraints.
As a result, some parties may suffer from insufficient data available for
training machine learning models. In this paper, we propose a multi-party dual
learning (MPDL) framework to alleviate the problem of limited data with poor
quality in an isolated party. Since the knowledge sharing processes for
multiple parties always emerge in dual forms, we show that dual learning is
naturally suitable to handle the challenge of missing data, and explicitly
exploits the probabilistic correlation and structural relationship between dual
tasks to regularize the training process. We introduce a feature-oriented
differential privacy with mathematical proof, in order to avoid possible
privacy leakage of raw features in the dual inference process. The approach
requires minimal modifications to the existing multi-party learning structure,
and each party can build flexible and powerful models separately, whose
accuracy is no less than non-distributed self-learning approaches. The MPDL
framework achieves significant improvement compared with state-of-the-art
multi-party learning methods, as we demonstrated through simulations on
real-world datasets.
