  Protein language models learn powerful representations directly from
sequences of amino acids. However, they are constrained to generate proteins
with only the set of amino acids represented in their vocabulary. In contrast,
chemical language models learn atom-level representations of smaller molecules
that include every atom, bond, and ring. In this work, we show that chemical
language models can learn atom-level representations of proteins enabling
protein generation unconstrained to the standard genetic code and far beyond
it. In doing so, we show that language models can generate entire proteins atom
by atom -- effectively learning the multiple hierarchical layers of molecular
information that define proteins from their primary sequence to their
secondary, and tertiary structure. We demonstrate language models are able to
explore beyond protein space -- generating proteins with modified sidechains
that form unnatural amino acids. Even further, we find that language models can
explore chemical space and protein space simultaneously and generate novel
examples of protein-drug conjugates. The results demonstrate the potential for
biomolecular design at the atom level using language models.
