  Transformers have shown promising progress in various visual object detection
tasks, including monocular 2D/3D detection and surround-view 3D detection. More
importantly, the attention mechanism in the Transformer model and the image
correspondence in binocular stereo are both similarity-based. However, directly
applying existing Transformer-based detectors to binocular stereo 3D object
detection leads to slow convergence and significant precision drops. We argue
that a key cause of this defect is that existing Transformers ignore the
stereo-specific image correspondence information. In this paper, we explore the
model design of Transformers in binocular 3D object detection, focusing
particularly on extracting and encoding the task-specific image correspondence
information. To achieve this goal, we present TS3D, a Transformer-based
Stereo-aware 3D object detector. In the TS3D, a Disparity-Aware Positional
Encoding (DAPE) module is proposed to embed the image correspondence
information into stereo features. The correspondence is encoded as normalized
sub-pixel-level disparity and is used in conjunction with sinusoidal 2D
positional encoding to provide the 3D location information of the scene. To
extract enriched multi-scale stereo features, we propose a Stereo Preserving
Feature Pyramid Network (SPFPN). The SPFPN is designed to preserve the
correspondence information while fusing intra-scale and aggregating cross-scale
stereo features. Our proposed TS3D achieves a 41.29% Moderate Car detection
average precision on the KITTI test set and takes 88 ms to detect objects from
each binocular image pair. It is competitive with advanced counterparts in
terms of both precision and inference speed.
