  Demonstrations are widely used in Deep Reinforcement Learning (DRL) for
facilitating solving tasks with sparse rewards. However, the tasks in
real-world scenarios can often have varied initial conditions from the
demonstration, which would require additional prior behaviours. For example,
consider we are given the demonstration for the task of \emph{picking up an
object from an open drawer}, but the drawer is closed in the training. Without
acquiring the prior behaviours of opening the drawer, the robot is unlikely to
solve the task. To address this, in this paper we propose an Intrinsic Rewards
Driven Example-based Control \textbf{(IRDEC)}. Our method can endow agents with
the ability to explore and acquire the required prior behaviours and then
connect to the task-specific behaviours in the demonstration to solve
sparse-reward tasks without requiring additional demonstration of the prior
behaviours. The performance of our method outperforms other baselines on three
navigation tasks and one robotic manipulation task with sparse rewards. Codes
are available at https://github.com/Ricky-Zhu/IRDEC.
