  Accurate and timely crop mapping is essential for yield estimation, insurance
claims, and conservation efforts. Over the years, many successful machine
learning models for crop mapping have been developed that use just the
multi-spectral imagery from satellites to predict crop type over the area of
interest. However, these traditional methods do not account for the physical
processes that govern crop growth. At a high level, crop growth can be
envisioned as physical parameters, such as weather and soil type, acting upon
the plant leading to crop growth which can be observed via satellites. In this
paper, we propose Weather-based Spatio-Temporal segmentation network with
ATTention (WSTATT), a deep learning model that leverages this understanding of
crop growth by formulating it as an inverse model that combines weather
(Daymet) and satellite imagery (Sentinel-2) to generate accurate crop maps. We
show that our approach provides significant improvements over existing
algorithms that solely rely on spectral imagery by comparing segmentation maps
and F1 classification scores. Furthermore, effective use of attention in WSTATT
architecture enables detection of crop types earlier in the season (up to 5
months in advance), which is very useful for improving food supply projections.
We finally discuss the impact of weather by correlating our results with crop
phenology to show that WSTATT is able to capture physical properties of crop
growth.
