  Thermal infrared (TIR) pedestrian tracking is one of the important components
among numerous applications of computer vision, which has a major advantage: it
can track pedestrians in total darkness. The ability to evaluate the TIR
pedestrian tracker fairly, on a benchmark dataset, is significant for the
development of this field. However, there is not a benchmark dataset. In this
paper, we develop a TIR pedestrian tracking dataset for the TIR pedestrian
tracker evaluation. The dataset includes 60 thermal sequences with manual
annotations. Each sequence has nine attribute labels for the attribute based
evaluation. In addition to the dataset, we carry out the large-scale evaluation
experiments on our benchmark dataset using nine publicly available trackers.
The experimental results help us understand the strengths and weaknesses of
these trackers.In addition, in order to gain more insight into the TIR
pedestrian tracker, we divide its functions into three components: feature
extractor, motion model, and observation model. Then, we conduct three
comparison experiments on our benchmark dataset to validate how each component
affects the tracker's performance. The findings of these experiments provide
some guidelines for future research. The dataset and evaluation toolkit can be
downloaded at {https://github.com/QiaoLiuHit/PTB-TIR_Evaluation_toolkit}.
