  Categorical Distributional Reinforcement Learning (CDRL) has demonstrated
superior sample efficiency in learning complex tasks compared to conventional
Reinforcement Learning (RL) approaches. However, the practical application of
CDRL is encumbered by challenging projection steps, detailed parameter tuning,
and domain knowledge. This paper addresses these challenges by introducing a
pioneering Continuous Distributional Model-Free RL algorithm tailored for
continuous action spaces. The proposed algorithm simplifies the implementation
of distributional RL, adopting an actor-critic architecture wherein the critic
outputs a continuous probability distribution. Additionally, we propose an
ensemble of multiple critics fused through a Kalman fusion mechanism to
mitigate overestimation bias. Through a series of experiments, we validate that
our proposed method is easy to train and serves as a sample-efficient solution
for executing complex continuous-control tasks.
