  Being able to interpret a machine learning model is a crucial task in many
applications of machine learning. Specifically, local interpretability is
important in determining why a model makes particular predictions. Despite the
recent focus on AI interpretability, there has been a lack of research in local
interpretability methods for time series forecasting while the few
interpretable methods that exist mainly focus on time series classification
tasks. In this study, we propose two novel evaluation metrics for time series
forecasting: Area Over the Perturbation Curve for Regression and Ablation
Percentage Threshold. These two metrics can measure the local fidelity of local
explanation models. We extend the theoretical foundation to collect
experimental results on two popular datasets, \textit{Rossmann sales} and
\textit{electricity}. Both metrics enable a comprehensive comparison of
numerous local explanation models and find which metrics are more sensitive.
Lastly, we provide heuristical reasoning for this analysis.
