  The Detection Transformer (DETR) has revolutionized the design of CNN-based
object detection systems, showcasing impressive performance. However, its
potential in the domain of multi-frame 3D object detection remains largely
unexplored. In this paper, we present STEMD, a novel end-to-end framework that
enhances the DETR-like paradigm for multi-frame 3D object detection by
addressing three key aspects specifically tailored for this task. First, to
model the inter-object spatial interaction and complex temporal dependencies,
we introduce the spatial-temporal graph attention network, which represents
queries as nodes in a graph and enables effective modeling of object
interactions within a social context. To solve the problem of missing hard
cases in the proposed output of the encoder in the current frame, we
incorporate the output of the previous frame to initialize the query input of
the decoder. Finally, it poses a challenge for the network to distinguish
between the positive query and other highly similar queries that are not the
best match. And similar queries are insufficiently suppressed and turn into
redundant prediction boxes. To address this issue, our proposed IoU
regularization term encourages similar queries to be distinct during the
refinement. Through extensive experiments, we demonstrate the effectiveness of
our approach in handling challenging scenarios, while incurring only a minor
additional computational overhead. The code is publicly available at
https://github.com/Eaphan/STEMD.
