  Deep learning in combination with improved training techniques and high
computational power has led to recent advances in the field of reinforcement
learning (RL) and to successful robotic RL applications such as in-hand
manipulation. However, most robotic RL relies on a well known initial state
distribution. In real-world tasks, this information is however often not
available. For example, when disentangling waste objects the actual position of
the robot w.r.t.\ the objects may not match the positions the RL policy was
trained for. To solve this problem, we present a novel adversarial
reinforcement learning (ARL) framework. The ARL framework utilizes an
adversary, which is trained to steer the original agent, the protagonist, to
challenging states. We train the protagonist and the adversary jointly to allow
them to adapt to the changing policy of their opponent. We show that our method
can generalize from training to test scenarios by training an end-to-end system
for robot control to solve a challenging object disentangling task. Experiments
with a KUKA LBR+ 7-DOF robot arm show that our approach outperforms the
baseline method in disentangling when starting from different initial states
than provided during training.
