  The privacy concern exists when the central server has the copies of
datasets. Hence, there is a paradigm shift for the learning networks to change
from centralized in-cloud learning to distributed \mbox{on-device} learning.
Benefit from the parallel computing, the on-device learning networks have a
lower bandwidth requirement than the in-cloud learning networks. Moreover, the
on-device learning networks also have several desirable characteristics such as
privacy preserving and flexibility. However, the \mbox{on-device} learning
networks are vulnerable to the malfunctioning terminals across the networks.
The worst-case malfunctioning terminals are the Byzantine adversaries, that can
perform arbitrary harmful operations to compromise the learned model based on
the full knowledge of the networks. Hence, the design of secure learning
algorithms becomes an emerging topic in the on-device learning networks with
Byzantine adversaries. In this article, we present a comprehensive overview of
the prevalent secure learning algorithms for the two promising on-device
learning networks: Federated-Learning networks and decentralized-learning
networks. We also review several future research directions in the
\mbox{Federated-Learning} and decentralized-learning networks.
