  Generalization capabilities, or rather a lack thereof, is one of the most
important unsolved problems in the field of robot learning, and while several
large scale efforts have set out to tackle this problem, unsolved it remains.
In this paper, we hypothesize that learning temporal action abstractions using
latent variable models (LVMs), which learn to map data to a compressed latent
space and back, is a promising direction towards low-level skills that can
readily be used for new tasks. Although several works have attempted to show
this, they have generally been limited by architectures that do not faithfully
capture shareable representations. To address this we present Quantized Skill
Transformer (QueST), which learns a larger and more flexible latent encoding
that is more capable of modeling the breadth of low-level skills necessary for
a variety of tasks. To make use of this extra flexibility, QueST imparts causal
inductive bias from the action sequence data into the latent space, leading to
more semantically useful and transferable representations. We compare to
state-of-the-art imitation learning and LVM baselines and see that QueST's
architecture leads to strong performance on several multitask and few-shot
learning benchmarks. Further results and videos are available at
https://quest-model.github.io/
