  Emerging Distributed AI systems are revolutionizing big data computing and
data processing capabilities with growing economic and societal impact.
However, recent studies have identified new attack surfaces and risks caused by
security, privacy, and fairness issues in AI systems. In this paper, we review
representative techniques, algorithms, and theoretical foundations for
trustworthy distributed AI through robustness guarantee, privacy protection,
and fairness awareness in distributed learning. We first provide a brief
overview of alternative architectures for distributed learning, discuss
inherent vulnerabilities for security, privacy, and fairness of AI algorithms
in distributed learning, and analyze why these problems are present in
distributed learning regardless of specific architectures. Then we provide a
unique taxonomy of countermeasures for trustworthy distributed AI, covering (1)
robustness to evasion attacks and irregular queries at inference, and
robustness to poisoning attacks, Byzantine attacks, and irregular data
distribution during training; (2) privacy protection during distributed
learning and model inference at deployment; and (3) AI fairness and governance
with respect to both data and models. We conclude with a discussion on open
challenges and future research directions toward trustworthy distributed AI,
such as the need for trustworthy AI policy guidelines, the AI
responsibility-utility co-design, and incentives and compliance.
