  Learning to estimate 3D geometry in a single image by watching unlabeled
videos via deep convolutional network is attracting significant attention. In
this paper, we introduce a "3D as-smooth-as-possible (3D-ASAP)" prior inside
the pipeline, which enables joint estimation of edges and 3D scene, yielding
results with significant improvement in accuracy for fine detailed structures.
Specifically, we define the 3D-ASAP prior by requiring that any two points
recovered in 3D from an image should lie on an existing planar surface if no
other cues provided. We design an unsupervised framework that Learns Edges and
Geometry (depth, normal) all at Once (LEGO). The predicted edges are embedded
into depth and surface normal smoothness terms, where pixels without edges
in-between are constrained to satisfy the prior. In our framework, the
predicted depths, normals and edges are forced to be consistent all the time.
We conduct experiments on KITTI to evaluate our estimated geometry and
CityScapes to perform edge evaluation. We show that in all of the tasks,
i.e.depth, normal and edge, our algorithm vastly outperforms other
state-of-the-art (SOTA) algorithms, demonstrating the benefits of our approach.
