  A generalist robot equipped with learned skills must be able to perform many
tasks in many different environments. However, zero-shot generalization to new
settings is not always possible. When the robot encounters a new environment or
object, it may need to finetune some of its previously learned skills to
accommodate this change. But crucially, previously learned behaviors and models
should still be suitable to accelerate this relearning. In this paper, we aim
to study how generative models of possible outcomes can allow a robot to learn
visual representations of affordances, so that the robot can sample potentially
possible outcomes in new situations, and then further train its policy to
achieve those outcomes. In effect, prior data is used to learn what kinds of
outcomes may be possible, such that when the robot encounters an unfamiliar
setting, it can sample potential outcomes from its model, attempt to reach
them, and thereby update both its skills and its outcome model. This approach,
visuomotor affordance learning (VAL), can be used to train goal-conditioned
policies that operate on raw image inputs, and can rapidly learn to manipulate
new objects via our proposed affordance-directed exploration scheme. We show
that VAL can utilize prior data to solve real-world tasks such drawer opening,
grasping, and placing objects in new scenes with only five minutes of online
experience in the new scene.
