  Decoder-only models, such as GPT, have demonstrated superior performance in
many areas compared to traditional encoder-decoder structure transformer
models. Over the years, end-to-end models based on the traditional transformer
structure, like MOTR, have achieved remarkable performance in multi-object
tracking. However, the significant computational resource consumption of these
models leads to less friendly inference speeds and training times. To address
these issues, this paper attempts to construct a lightweight Decoder-only
model: DecoderTracker for end-to-end multi-object tracking. Specifically,
drawing on some real-time detection models, we have developed an image feature
extraction network which can efficiently extract features from images to
replace the encoder structure. In addition to minor innovations in the network,
we analyze the potential reasons for the slow training of MOTR-like models and
propose an effective training strategy to mitigate the issue of prolonged
training times. On the DanceTrack dataset, without any bells and whistles,
DecoderTracker's tracking performance slightly surpasses that of MOTR, with
approximately twice the inference speed. Furthermore, DecoderTracker requires
significantly less training time compared to MOTR.
