  Automated crop-type classification using Sentinel-2 satellite time series is
essential to support agriculture monitoring. Recently, deep learning models
based on transformer encoders became a promising approach for crop-type
classification. Using explainable machine learning to reveal the inner workings
of these models is an important step towards improving stakeholders' trust and
efficient agriculture monitoring.
  In this paper, we introduce a novel explainability framework that aims to
shed a light on the essential crop disambiguation patterns learned by a
state-of-the-art transformer encoder model. More specifically, we process the
attention weights of a trained transformer encoder to reveal the critical dates
for crop disambiguation and use domain knowledge to uncover the phenological
events that support the model performance. We also present a sensitivity
analysis approach to understand better the attention capability for revealing
crop-specific phenological events.
  We report compelling results showing that attention patterns strongly relate
to key dates, and consequently, to the critical phenological events for
crop-type classification. These findings might be relevant for improving
stakeholder trust and optimizing agriculture monitoring processes.
Additionally, our sensitivity analysis demonstrates the limitation of attention
weights for identifying the important events in the crop phenology as we
empirically show that the unveiled phenological events depend on the other
crops in the data considered during training.
