  This paper studies the complex task of simultaneous multi-object 3D
reconstruction, 6D pose and size estimation from a single-view RGB-D
observation. In contrast to instance-level pose estimation, we focus on a more
challenging problem where CAD models are not available at inference time.
Existing approaches mainly follow a complex multi-stage pipeline which first
localizes and detects each object instance in the image and then regresses to
either their 3D meshes or 6D poses. These approaches suffer from
high-computational cost and low performance in complex multi-object scenarios,
where occlusions can be present. Hence, we present a simple one-stage approach
to predict both the 3D shape and estimate the 6D pose and size jointly in a
bounding-box free manner. In particular, our method treats object instances as
spatial centers where each center denotes the complete shape of an object along
with its 6D pose and size. Through this per-pixel representation, our approach
can reconstruct in real-time (40 FPS) multiple novel object instances and
predict their 6D pose and sizes in a single-forward pass. Through extensive
experiments, we demonstrate that our approach significantly outperforms all
shape completion and categorical 6D pose and size estimation baselines on
multi-object ShapeNet and NOCS datasets respectively with a 12.6% absolute
improvement in mAP for 6D pose for novel real-world object instances.
