  The application of reinforcement learning (RL) in robotic control is still
limited in the environments with sparse and delayed rewards. In this paper, we
propose a practical self-imitation learning method named Self-Imitation
Learning with Constant Reward (SILCR). Instead of requiring hand-defined
immediate rewards from environments, our method assigns the immediate rewards
at each timestep with constant values according to their final episodic
rewards. In this way, even if the dense rewards from environments are
unavailable, every action taken by the agents would be guided properly. We
demonstrate the effectiveness of our method in some challenging continuous
robotics control tasks in MuJoCo simulation and the results show that our
method significantly outperforms the alternative methods in tasks with sparse
and delayed rewards. Even compared with alternatives with dense rewards
available, our method achieves competitive performance. The ablation
experiments also show the stability and reproducibility of our method.
