  Heterogeneous big data poses many challenges in machine learning. Its
enormous scale, high dimensionality, and inherent uncertainty make almost every
aspect of machine learning difficult, from providing enough processing power to
maintaining model accuracy to protecting privacy. However, perhaps the most
imposing problem is that big data is often interspersed with sensitive personal
data. Hence, we propose a privacy-preserving hierarchical fuzzy neural network
(PP-HFNN) to address these technical challenges while also alleviating privacy
concerns. The network is trained with a two-stage optimization algorithm, and
the parameters at low levels of the hierarchy are learned with a scheme based
on the well-known alternating direction method of multipliers, which does not
reveal local data to other agents. Coordination at high levels of the hierarchy
is handled by the alternating optimization method, which converges very
quickly. The entire training procedure is scalable, fast and does not suffer
from gradient vanishing problems like the methods based on back-propagation.
Comprehensive simulations conducted on both regression and classification tasks
demonstrate the effectiveness of the proposed model.
