  Crop yield forecasting plays a significant role in addressing growing
concerns about food security and guiding decision-making for policymakers and
farmers. When deep learning is employed, understanding the learning and
decision-making processes of the models, as well as their interaction with the
input data, is crucial for establishing trust in the models and gaining insight
into their reliability. In this study, we focus on the task of crop yield
prediction, specifically for soybean, wheat, and rapeseed crops in Argentina,
Uruguay, and Germany. Our goal is to develop and explain predictive models for
these crops, using a large dataset of satellite images, additional data
modalities, and crop yield maps. We employ a long short-term memory network and
investigate the impact of using different temporal samplings of the satellite
data and the benefit of adding more relevant modalities. For model
explainability, we utilize feature attribution methods to quantify input
feature contributions, identify critical growth stages, analyze yield
variability at the field level, and explain less accurate predictions.
  The modeling results show an improvement when adding more modalities or using
all available instances of satellite data. The explainability results reveal
distinct feature importance patterns for each crop and region. We further found
that the most influential growth stages on the prediction are dependent on the
temporal sampling of the input data. We demonstrated how these critical growth
stages, which hold significant agronomic value, closely align with the existing
literature in agronomy and crop development biology.
