  3D reconstruction aims to reconstruct 3D objects from 2D views. Previous
works for 3D reconstruction mainly focus on feature matching between views or
using CNNs as backbones. Recently, Transformers have been shown effective in
multiple applications of computer vision. However, whether or not Transformers
can be used for 3D reconstruction is still unclear. In this paper, we fill this
gap by proposing 3D-RETR, which is able to perform end-to-end 3D REconstruction
with TRansformers. 3D-RETR first uses a pretrained Transformer to extract
visual features from 2D input images. 3D-RETR then uses another Transformer
Decoder to obtain the voxel features. A CNN Decoder then takes as input the
voxel features to obtain the reconstructed objects. 3D-RETR is capable of 3D
reconstruction from a single view or multiple views. Experimental results on
two datasets show that 3DRETR reaches state-of-the-art performance on 3D
reconstruction. Additional ablation study also demonstrates that 3D-DETR
benefits from using Transformers.
