  Due to the high similarity of disparity between consecutive frames in video
sequences, the area where disparity changes is defined as the residual map,
which can be calculated. Based on this, we propose RecSM, a network based on
residual estimation with a flexible recursive structure for video stereo
matching. The RecSM network accelerates stereo matching using a Multi-scale
Residual Estimation Module (MREM), which employs the temporal context as a
reference and rapidly calculates the disparity for the current frame by
computing only the residual values between the current and previous frames. To
further reduce the error of estimated disparities, we use the Disparity
Optimization Module (DOM) and Temporal Attention Module (TAM) to enforce
constraints between each module, and together with MREM, form a flexible
Stackable Computation Structure (SCS), which allows for the design of different
numbers of SCS based on practical scenarios. Experimental results demonstrate
that with a stack count of 3, RecSM achieves a 4x speed improvement compared to
ACVNet, running at 0.054 seconds based on one NVIDIA RTX 2080TI GPU, with an
accuracy decrease of only 0.7%. Code is available at
https://github.com/Y0uchenZ/RecSM.
