  Robots cannot yet match humans' ability to rapidly learn the shapes of novel
3D objects and recognize them robustly despite clutter and occlusion. We
present Bayes3D, an uncertainty-aware perception system for structured 3D
scenes, that reports accurate posterior uncertainty over 3D object shape, pose,
and scene composition in the presence of clutter and occlusion. Bayes3D
delivers these capabilities via a novel hierarchical Bayesian model for 3D
scenes and a GPU-accelerated coarse-to-fine sequential Monte Carlo algorithm.
Quantitative experiments show that Bayes3D can learn 3D models of novel objects
from just a handful of views, recognizing them more robustly and with orders of
magnitude less training data than neural baselines, and tracking 3D objects
faster than real time on a single GPU. We also demonstrate that Bayes3D learns
complex 3D object models and accurately infers 3D scene composition when used
on a Panda robot in a tabletop scenario.
