  Genomic selection (GS), as a critical crop breeding strategy, plays a key
role in enhancing food production and addressing the global hunger crisis. The
predominant approaches in GS currently revolve around employing statistical
methods for prediction. However, statistical methods often come with two main
limitations: strong statistical priors and linear assumptions. A recent trend
is to capture the non-linear relationships between markers by deep learning.
However, as crop datasets are commonly long sequences with limited samples, the
robustness of deep learning models, especially Transformers, remains a
challenge. In this work, to unleash the unexplored potential of attention
mechanism for the task of interest, we propose a simple yet effective
Transformer-based framework that enables end-to-end training of the whole
sequence. Via experiments on rice3k and wheat3k datasets, we show that, with
simple tricks such as k-mer tokenization and random masking, Transformer can
achieve overall superior performance against seminal methods on GS tasks of
interest.
