  Non-prehensile pushing actions have the potential to singulate a target
object from its surrounding clutter in order to facilitate the robotic grasping
of the target. To address this problem we utilize a heuristic rule that moves
the target object towards the workspace's empty space and demonstrate that this
simple heuristic rule achieves singulation. We incorporate this effective
heuristic rule to the reward in order to train more efficiently reinforcement
learning (RL) agents for singulation. Simulation experiments demonstrate that
this insight increases performance. Finally, our results show that the RL-based
policy implicitly learns something similar to one of the used heuristics in
terms of decision making. Qualitative results, code, pre-trained models and
simulation environments are available at
https://github.com/robot-clutter/improved_rl.
