  Learning control policies offline from pre-recorded datasets is a promising
avenue for solving challenging real-world problems. However, available datasets
are typically of mixed quality, with a limited number of the trajectories that
we would consider as positive examples; i.e., high-quality demonstrations.
Therefore, we propose a novel iterative learning algorithm for identifying
expert trajectories in unlabeled mixed-quality robotics datasets given a
minimal set of positive examples, surpassing existing algorithms in terms of
accuracy. We show that applying behavioral cloning to the resulting filtered
dataset outperforms several competitive offline reinforcement learning and
imitation learning baselines. We perform experiments on a range of simulated
locomotion tasks and on two challenging manipulation tasks on a real robotic
system; in these experiments, our method showcases state-of-the-art
performance. Our website:
\url{https://sites.google.com/view/offline-policy-learning-pubc}.
