  In order to collaborate efficiently with unknown partners in cooperative
control settings, adaptation of the partners based on online experience is
required. The rather general and widely applicable control setting, where each
cooperation partner might strive for individual goals while the control laws
and objectives of the partners are unknown, entails various challenges such as
the non-stationarity of the environment, the multi-agent credit assignment
problem, the alter-exploration problem and the coordination problem. We propose
new, modular deep decentralized Multi-Agent Reinforcement Learning mechanisms
to account for these challenges. Therefore, our method uses a time-dependent
prioritization of samples, incorporates a model of the system dynamics and
utilizes variable, accountability-driven learning rates and simulated,
artificial experiences in order to guide the learning process. The
effectiveness of our method is demonstrated by means of a simulated, nonlinear
cooperative control task.
