  Infrared small target detection (IRSTD) is a challenging task in computer
vision. During the last two decades, researchers' efforts are devoted to
improving detection ability of IRSTDs. Despite the huge improvement in
designing new algorithms, lack of extensive investigation of the evaluation
metrics are evident. Therefore, in this paper, a systematic approach is
utilized to: First, investigate the evaluation ability of current metrics;
Second, propose new evaluation metrics to address shortcoming of common
metrics. To this end, after carefully reviewing the problem, the required
conditions to have a successful detection are analyzed. Then, the shortcomings
of current evaluation metrics which include pre-thresholding as well as
post-thresholding metrics are determined. Based on the requirements of
real-world systems, new metrics are proposed. Finally, the proposed metrics are
used to compare and evaluate four well-known small infrared target detection
algorithms. The results show that new metrics are consistent with qualitative
results.
