  Devices at the edge of wireless networks are the last mile data sources for
machine learning (ML). As opposed to traditional ready-made public datasets,
these user-generated private datasets reflect the freshest local environments
in real time. They are thus indispensable for enabling mission-critical
intelligent systems, ranging from fog radio access networks (RANs) to
driverless cars and e-Health wearables. This article focuses on how to distill
high-quality on-device ML models using fog computing, from such user-generated
private data dispersed across wirelessly connected devices. To this end, we
introduce communication-efficient and privacy-preserving distributed ML
frameworks, termed fog ML (FML), wherein on-device ML models are trained by
exchanging model parameters, model outputs, and surrogate data. We then present
advanced FML frameworks addressing wireless RAN characteristics, limited
on-device resources, and imbalanced data distributions. Our study suggests that
the full potential of FML can be reached by co-designing communication and
distributed ML operations while accounting for heterogeneous hardware
specifications, data characteristics, and user requirements.
