  The deep multi-view stereo (MVS) and stereo matching approaches generally
construct 3D cost volumes to regularize and regress the output depth or
disparity. These methods are limited when high-resolution outputs are needed
since the memory and time costs grow cubically as the volume resolution
increases. In this paper, we propose a both memory and time efficient cost
volume formulation that is complementary to existing multi-view stereo and
stereo matching approaches based on 3D cost volumes. First, the proposed cost
volume is built upon a standard feature pyramid encoding geometry and context
at gradually finer scales. Then, we can narrow the depth (or disparity) range
of each stage by the depth (or disparity) map from the previous stage. With
gradually higher cost volume resolution and adaptive adjustment of depth (or
disparity) intervals, the output is recovered in a coarser to fine manner.
  We apply the cascade cost volume to the representative MVS-Net, and obtain a
23.1% improvement on DTU benchmark (1st place), with 50.6% and 74.2% reduction
in GPU memory and run-time. It is also the state-of-the-art learning-based
method on Tanks and Temples benchmark. The statistics of accuracy, run-time and
GPU memory on other representative stereo CNNs also validate the effectiveness
of our proposed method.
