  Learning in high dimensional continuous tasks is challenging, mainly when the
experience replay memory is very limited. We introduce a simple yet effective
experience sharing mechanism for deterministic policies in continuous action
domains for the future off-policy deep reinforcement learning applications in
which the allocated memory for the experience replay buffer is limited. To
overcome the extrapolation error induced by learning from other agents'
experiences, we facilitate our algorithm with a novel off-policy correction
technique without any action probability estimates. We test the effectiveness
of our method in challenging OpenAI Gym continuous control tasks and conclude
that it can achieve a safe experience sharing across multiple agents and
exhibits a robust performance when the replay memory is strictly limited.
