  We address the problem of safely solving complex bimanual robot manipulation
tasks with sparse rewards. Such challenging tasks can be decomposed into
sub-tasks that are accomplishable by different robots concurrently or
sequentially for better efficiency. While previous reinforcement learning
approaches primarily focus on modeling the compositionality of sub-tasks, two
fundamental issues are largely ignored particularly when learning cooperative
strategies for two robots: (i) domination, i.e., one robot may try to solve a
task by itself and leaves the other idle; (ii) conflict, i.e., one robot can
interrupt another's workspace when executing different sub-tasks
simultaneously, which leads to unsafe collisions. To tackle these two issues,
we propose a novel technique called disentangled attention, which provides an
intrinsic regularization for two robots to focus on separate sub-tasks and
objects. We evaluate our method on five bimanual manipulation tasks.
Experimental results show that our proposed intrinsic regularization
successfully avoids domination and reduces conflicts for the policies, which
leads to significantly more efficient and safer cooperative strategies than all
the baselines. Our project page with videos is at
https://mehooz.github.io/bimanual-attention.
