  Potts models and variational autoencoders (VAEs) have recently gained
popularity as generative protein sequence models (GPSMs) to explore fitness
landscapes and predict the effect of mutations. Despite encouraging results,
quantitative characterization and comparison of GPSM-generated probability
distributions is still lacking. It is currently unclear whether GPSMs can
faithfully reproduce the complex multi-residue mutation patterns observed in
natural sequences arising due to epistasis. We develop a set of sequence
statistics to assess the "generative capacity" of three GPSMs of recent
interest: the pairwise Potts Hamiltonian, the VAE, and the site-independent
model, using natural and synthetic datasets. We show that the generative
capacity of the Potts Hamiltonian model is the largest, in that the higher
order mutational statistics generated by the model agree with those observed
for natural sequences. In contrast, we show that the VAE's generative capacity
lies between the pairwise Potts and site-independent models. Importantly, our
work measures GPSM generative capacity in terms of higher-order sequence
covariation statistics which we have developed, and provides a new framework
for evaluating and interpreting GPSM accuracy that emphasizes the role of
epistasis.
