  We present the MSA-to-protein transformer, a generative model of protein
sequences conditioned on protein families represented by multiple sequence
alignments (MSAs). Unlike existing approaches to learning generative models of
protein families, the MSA-to-protein transformer conditions sequence generation
directly on a learned encoding of the multiple sequence alignment,
circumventing the need for fitting dedicated family models. By training on a
large set of well-curated multiple sequence alignments in Pfam, our
MSA-to-protein transformer generalizes well to protein families not observed
during training and outperforms conventional family modeling approaches,
especially when MSAs are small. Our generative approach accurately models
epistasis and indels and allows for exact inference and efficient sampling
unlike other approaches. We demonstrate the protein sequence modeling
capabilities of our MSA-to-protein transformer and compare it with alternative
sequence modeling approaches in comprehensive benchmark experiments.
