  In the field, robots often need to operate in unknown and unstructured
environments, where accurate sensing and state estimation (SE) becomes a major
challenge. Cameras have been used to great success in mapping and planning in
such environments, as well as complex but quasi-static tasks such as grasping,
but are rarely integrated into the control loop for unstable systems. Learning
pixel-to-torque control promises to allow robots to flexibly handle a wider
variety of tasks. Although they do not present additional theoretical
obstacles, learning pixel-to-torque control for unstable systems that that
require precise and high bandwidth control still poses a significant practical
challenge, and best practices have not yet been established. To help drive
reproducible research on the practical aspects of learning pixel-to-torque
control, we propose a platform that can flexibly represent the entire process,
from lab to deployment, for learning pixel-to-torque control on a robot with
fast, unstable dynamics: the vision-based Furuta pendulum. The platform can be
reproduced with either off-the-shelf or custom-built hardware. We expect that
this platform will allow researchers to quickly and systematically test
different approaches, as well as reproduce and benchmark case studies from
other labs. We also present a first case study on this system using DNNs which,
to the best of our knowledge, is the first demonstration of learning
pixel-to-torque control on an unstable system with update rates faster than 100
Hz. A video synopsis can be found online at https://youtu.be/S2llScfG-8E, and
in the supplementary material.
