  The main goal of this paper is to investigate continuous-time distributed
dynamic programming (DP) algorithms for networked multi-agent Markov decision
problems (MAMDPs). In our study, we adopt a distributed multi-agent framework
where individual agents have access only to their own rewards, lacking insights
into the rewards of other agents. Moreover, each agent has the ability to share
its parameters with neighboring agents through a communication network,
represented by a graph. We first introduce a novel distributed DP, inspired by
the distributed optimization method of Wang and Elia. Next, a new distributed
DP is introduced through a decoupling process. The convergence of the DP
algorithms is proved through systems and control perspectives. The study in
this paper sets the stage for new distributed temporal different learning
algorithms.
