  Recent temporal LiDAR-based 3D object detectors achieve promising performance
based on the two-stage proposal-based approach. They generate 3D box candidates
from the first-stage dense detector, followed by different temporal aggregation
methods. However, these approaches require per-frame objects or whole point
clouds, posing challenges related to memory bank utilization. Moreover, point
clouds and trajectory features are combined solely based on concatenation,
which may neglect effective interactions between them. In this paper, we
propose a point-trajectory transformer with long short-term memory for
efficient temporal 3D object detection. To this end, we only utilize point
clouds of current-frame objects and their historical trajectories as input to
minimize the memory bank storage requirement. Furthermore, we introduce modules
to encode trajectory features, focusing on long short-term and future-aware
perspectives, and then effectively aggregate them with point cloud features. We
conduct extensive experiments on the large-scale Waymo dataset to demonstrate
that our approach performs well against state-of-the-art methods. Code and
models will be made publicly available at https://github.com/kuanchihhuang/PTT.
