  Reinforcement Learning is a promising tool for learning complex policies even
in fast-moving and object-interactive domains where human teleoperation or
hard-coded policies might fail. To effectively reflect this challenging
category of tasks, we introduce a dynamic, interactive RL testbed based on
robot air hockey. By augmenting air hockey with a large family of tasks ranging
from easy tasks like reaching, to challenging ones like pushing a block by
hitting it with a puck, as well as goal-based and human-interactive tasks, our
testbed allows a varied assessment of RL capabilities. The robot air hockey
testbed also supports sim-to-real transfer with three domains: two simulators
of increasing fidelity and a real robot system. Using a dataset of
demonstration data gathered through two teleoperation systems: a virtualized
control environment, and human shadowing, we assess the testbed with behavior
cloning, offline RL, and RL from scratch.
