  Large amount of data is often required to train and deploy useful machine
learning models in industry. Smaller enterprises do not have the luxury of
accessing enough data for machine learning, For privacy sensitive fields such
as banking, insurance and healthcare, aggregating data to a data warehouse
poses a challenge of data security and limited computational resources. These
challenges are critical when developing machine learning algorithms in
industry. Several attempts have been made to address the above challenges by
using distributed learning techniques such as federated learning over disparate
data stores in order to circumvent the need for centralised data aggregation.
This paper proposes an improved algorithm to securely train deep neural
networks over several data sources in a distributed way, in order to eliminate
the need to centrally aggregate the data and the need to share the data thus
preserving privacy. The proposed method allows training of deep neural networks
using data from multiple de-linked nodes in a distributed environment and to
secure the representation shared during training. Only a representation of the
trained models (network architecture and weights) are shared. The algorithm was
evaluated on existing healthcare patients data and the performance of this
implementation was compared to that of a regular deep neural network trained on
a single centralised architecture. This algorithm will pave a way for
distributed training of neural networks on privacy sensitive applications where
raw data may not be shared directly or centrally aggregating this data in a
data warehouse is not feasible.
