  Physical interactions can often help reveal information that is not readily
apparent. For example, we may tug at a table leg to evaluate whether it is
built well, or turn a water bottle upside down to check that it is watertight.
We propose to train robots to acquire such interactive behaviors automatically,
for the purpose of evaluating the result of an attempted robotic skill
execution. These evaluations in turn serve as "interactive reward functions"
(IRFs) for training reinforcement learning policies to perform the target
skill, such as screwing the table leg tightly. In addition, even after task
policies are fully trained, IRFs can serve as verification mechanisms that
improve online task execution. For any given task, our IRFs can be conveniently
trained using only examples of successful outcomes, and no further
specification is needed to train the task policy thereafter. In our evaluations
on door locking and weighted block stacking in simulation, and screw tightening
on a real robot, IRFs enable large performance improvements, even outperforming
baselines with access to demonstrations or carefully engineered rewards.
Project website: https://sites.google.com/view/lirf-corl-2022/
