  This paper proposes the Cooperative Soft Actor Critic (CSAC) method of
enabling consecutive reinforcement learning agents to cooperatively solve a
long time horizon multi-stage task. This method is achieved by modifying the
policy of each agent to maximise both the current and next agent's critic.
Cooperatively maximising each agent's critic allows each agent to take actions
that are beneficial for its task as well as subsequent tasks. Using this method
in a multi-room maze domain, the cooperative policies were able to outperform
both uncooperative policies as well as a single agent trained across the entire
domain. CSAC achieved a success rate of at least 20\% higher than the
uncooperative policies, and converged on a solution at least 4 times faster
than the single agent.
