  We present a framework for data-driven robotics that makes use of a large
dataset of recorded robot experience and scales to several tasks using learned
reward functions. We show how to apply this framework to accomplish three
different object manipulation tasks on a real robot platform. Given
demonstrations of a task together with task-agnostic recorded experience, we
use a special form of human annotation as supervision to learn a reward
function, which enables us to deal with real-world tasks where the reward
signal cannot be acquired directly. Learned rewards are used in combination
with a large dataset of experience from different tasks to learn a robot policy
offline using batch RL. We show that using our approach it is possible to train
agents to perform a variety of challenging manipulation tasks including
stacking rigid objects and handling cloth.
