  To endow machines with the ability to perceive the real-world in a three
dimensional representation as we do as humans is a fundamental and
long-standing topic in Artificial Intelligence. Given different types of visual
inputs such as images or point clouds acquired by 2D/3D sensors, one important
goal is to understand the geometric structure and semantics of the 3D
environment. Traditional approaches usually leverage hand-crafted features to
estimate the shape and semantics of objects or scenes. However, they are
difficult to generalize to novel objects and scenarios, and struggle to
overcome critical issues caused by visual occlusions. By contrast, we aim to
understand scenes and the objects within them by learning general and robust
representations using deep neural networks, trained on large-scale real-world
3D data. To achieve these aims, this thesis makes three core contributions from
object-level 3D shape estimation from single or multiple views to scene-level
semantic understanding.
