  We propose a new visual hierarchical representation paradigm for multi-object
tracking. It is more effective to discriminate between objects by attending to
objects' compositional visual regions and contrasting with the background
contextual information instead of sticking to only the semantic visual cue such
as bounding boxes. This compositional-semantic-contextual hierarchy is flexible
to be integrated in different appearance-based multi-object tracking methods.
We also propose an attention-based visual feature module to fuse the
hierarchical visual representations. The proposed method achieves
state-of-the-art accuracy and time efficiency among query-based methods on
multiple multi-object tracking benchmarks.
