  Despite the recent progress in video understanding and the continuous rate of
improvement in temporal action localization throughout the years, it is still
unclear how far (or close?) we are to solving the problem. To this end, we
introduce a new diagnostic tool to analyze the performance of temporal action
detectors in videos and compare different methods beyond a single scalar
metric. We exemplify the use of our tool by analyzing the performance of the
top rewarded entries in the latest ActivityNet action localization challenge.
Our analysis shows that the most impactful areas to work on are: strategies to
better handle temporal context around the instances, improving the robustness
w.r.t. the instance absolute and relative size, and strategies to reduce the
localization errors. Moreover, our experimental analysis finds the lack of
agreement among annotator is not a major roadblock to attain progress in the
field. Our diagnostic tool is publicly available to keep fueling the minds of
other researchers with additional insights about their algorithms.
