  In this work we investigate and demonstrate benefits of a Bayesian approach
to imitation learning from multiple sensor inputs, as applied to the task of
opening office doors with a mobile manipulator. Augmenting policies with
additional sensor inputs, such as RGB + depth cameras, is a straightforward
approach to improving robot perception capabilities, especially for tasks that
may favor different sensors in different situations. As we scale multi-sensor
robotic learning to unstructured real-world settings (e.g. offices, homes) and
more complex robot behaviors, we also increase reliance on simulators for cost,
efficiency, and safety. Consequently, the sim-to-real gap across multiple
sensor modalities also increases, making simulated validation more difficult.
We show that using the Variational Information Bottleneck (Alemi et al., 2016)
to regularize convolutional neural networks improves generalization to held-out
domains and reduces the sim-to-real gap in a sensor-agnostic manner. As a side
effect, the learned embeddings also provide useful estimates of model
uncertainty for each sensor. We demonstrate that our method is able to help
close the sim-to-real gap and successfully fuse RGB and depth modalities based
on understanding of the situational uncertainty of each sensor. In a real-world
office environment, we achieve 96% task success, improving upon the baseline by
+16%.
