  This paper presents a comprehensive survey of Federated Reinforcement
Learning (FRL), an emerging and promising field in Reinforcement Learning (RL).
Starting with a tutorial of Federated Learning (FL) and RL, we then focus on
the introduction of FRL as a new method with great potential by leveraging the
basic idea of FL to improve the performance of RL while preserving
data-privacy. According to the distribution characteristics of the agents in
the framework, FRL algorithms can be divided into two categories, i.e.
Horizontal Federated Reinforcement Learning (HFRL) and Vertical Federated
Reinforcement Learning (VFRL). We provide the detailed definitions of each
category by formulas, investigate the evolution of FRL from a technical
perspective, and highlight its advantages over previous RL algorithms. In
addition, the existing works on FRL are summarized by application fields,
including edge computing, communication, control optimization, and attack
detection. Finally, we describe and discuss several key research directions
that are crucial to solving the open problems within FRL.
