  Various machine-learning models, including deep neural network models, have
already been developed to predict deleteriousness of missense (non-synonymous)
mutations. Potential improvements to the current state of the art, however, may
still benefit from a fresh look at the biological problem using more
sophisticated self-adaptive machine-learning approaches. Recent advances in the
natural language processing field show transformer models-a type of deep neural
network-to be particularly powerful at modeling sequence information with
context dependence. In this study, we introduce MutFormer, a transformer-based
model for the prediction of deleterious missense mutations, which uses
reference and mutated protein sequences from the human genome as the primary
features. MutFormer takes advantage of a combination of self-attention layers
and convolutional layers to learn both long-range and short-range dependencies
between amino acid mutations in a protein sequence. In this study, we first
pre-trained MutFormer on reference protein sequences and mutated protein
sequences resulting from common genetic variants observed in human populations.
We next examined different fine-tuning methods to successfully apply the model
to deleteriousness prediction of missense mutations. Finally, we evaluated
MutFormer's performance on multiple testing data sets. We found that MutFormer
showed similar or improved performance over a variety of existing tools,
including those that used conventional machine-learning approaches. We conclude
that MutFormer successfully considers sequence features that are not explored
in previous studies and could potentially complement existing computational
predictions or empirically generated functional scores to improve our
understanding of disease variants.
