  Enabling robots to solve multiple manipulation tasks has a wide range of
industrial applications. While learning-based approaches enjoy flexibility and
generalizability, scaling these approaches to solve such compositional tasks
remains a challenge. In this work, we aim to solve multi-task learning through
the lens of sequence-conditioning and weighted sampling. First, we propose a
new suite of benchmark specifically aimed at compositional tasks, MultiRavens,
which allows defining custom task combinations through task modules that are
inspired by industrial tasks and exemplify the difficulties in vision-based
learning and planning methods. Second, we propose a vision-based end-to-end
system architecture, Sequence-Conditioned Transporter Networks, which augments
Goal-Conditioned Transporter Networks with sequence-conditioning and weighted
sampling and can efficiently learn to solve multi-task long horizon problems.
Our analysis suggests that not only the new framework significantly improves
pick-and-place performance on novel 10 multi-task benchmark problems, but also
the multi-task learning with weighted sampling can vastly improve learning and
agent performances on individual tasks.
