  Time series are ubiquitous, and a measure to assess their similarity is a
core part of many computational systems. In particular, the similarity measure
is the most essential ingredient of time series clustering and classification
systems. Because of this importance, countless approaches to estimate time
series similarity have been proposed. However, there is a lack of comparative
studies using empirical, rigorous, quantitative, and large-scale assessment
strategies. In this article, we provide an extensive evaluation of similarity
measures for time series classification following the aforementioned
principles. We consider 7 different measures coming from alternative measure
`families', and 45 publicly-available time series data sets coming from a wide
variety of scientific domains. We focus on out-of-sample classification
accuracy, but in-sample accuracies and parameter choices are also discussed.
Our work is based on rigorous evaluation methodologies and includes the use of
powerful statistical significance tests to derive meaningful conclusions. The
obtained results show the equivalence, in terms of accuracy, of a number of
measures, but with one single candidate outperforming the rest. Such findings,
together with the followed methodology, invite researchers on the field to
adopt a more consistent evaluation criteria and a more informed decision
regarding the baseline measures to which new developments should be compared.
