  We present a method for jointly predicting a depth map and intrinsic images
from single-image input. The two tasks are formulated in a synergistic manner
through a joint conditional random field (CRF) that is solved using a novel
convolutional neural network (CNN) architecture, called the joint convolutional
neural field (JCNF) model. Tailored to our joint estimation problem, JCNF
differs from previous CNNs in its sharing of convolutional activations and
layers between networks for each task, its inference in the gradient domain
where there exists greater correlation between depth and intrinsic images, and
the incorporation of a gradient scale network that learns the confidence of
estimated gradients in order to effectively balance them in the solution. This
approach is shown to surpass state-of-the-art methods both on single-image
depth estimation and on intrinsic image decomposition.
