  Imitation learning is a powerful machine learning algorithm for a robot to
acquire manipulation skills. Nevertheless, many real-world manipulation tasks
involve precise and dexterous robot-object interactions, which make it
difficult for humans to collect high-quality expert demonstrations. As a
result, a robot has to learn skills from suboptimal demonstrations and
unstructured interactions, which remains a key challenge. Existing works
typically use offline deep reinforcement learning (RL) to solve this challenge,
but in practice these algorithms are unstable and fragile due to the deadly
triad issue. To overcome this problem, we propose GSR, a simple yet effective
algorithm that learns from suboptimal demonstrations through Graph Search and
Retrieval. We first use pretrained representation to organize the interaction
experience into a graph and perform a graph search to calculate the values of
different behaviors. Then, we apply a retrieval-based procedure to identify the
best behavior (actions) on each state and use behavior cloning to learn that
behavior. We evaluate our method in both simulation and real-world robotic
manipulation tasks with complex visual inputs, covering various precise and
dexterous manipulation skills with objects of different physical properties.
GSR can achieve a 10% to 30% higher success rate and over 30% higher
proficiency compared to baselines. Our project page is at
https://zhaohengyin.github.io/gsr.
