  Estimating the accurate depth from a single image is challenging since it is
inherently ambiguous and ill-posed. While recent works design increasingly
complicated and powerful networks to directly regress the depth map, we take
the path of CRFs optimization. Due to the expensive computation, CRFs are
usually performed between neighborhoods rather than the whole graph. To
leverage the potential of fully-connected CRFs, we split the input into windows
and perform the FC-CRFs optimization within each window, which reduces the
computation complexity and makes FC-CRFs feasible. To better capture the
relationships between nodes in the graph, we exploit the multi-head attention
mechanism to compute a multi-head potential function, which is fed to the
networks to output an optimized depth map. Then we build a bottom-up-top-down
structure, where this neural window FC-CRFs module serves as the decoder, and a
vision transformer serves as the encoder. The experiments demonstrate that our
method significantly improves the performance across all metrics on both the
KITTI and NYUv2 datasets, compared to previous methods. Furthermore, the
proposed method can be directly applied to panorama images and outperforms all
previous panorama methods on the MatterPort3D dataset. Project page:
https://weihaosky.github.io/newcrfs.
