  How can we explain the predictions of a machine learning model? When the data
is structured as a multivariate time series, this question induces additional
difficulties such as the necessity for the explanation to embody the time
dependency and the large number of inputs. To address these challenges, we
propose dynamic masks (Dynamask). This method produces instance-wise importance
scores for each feature at each time step by fitting a perturbation mask to the
input sequence. In order to incorporate the time dependency of the data,
Dynamask studies the effects of dynamic perturbation operators. In order to
tackle the large number of inputs, we propose a scheme to make the feature
selection parsimonious (to select no more feature than necessary) and legible
(a notion that we detail by making a parallel with information theory). With
synthetic and real-world data, we demonstrate that the dynamic underpinning of
Dynamask, together with its parsimony, offer a neat improvement in the
identification of feature importance over time. The modularity of Dynamask
makes it ideal as a plug-in to increase the transparency of a wide range of
machine learning models in areas such as medicine and finance, where time
series are abundant.
