  Inferring the structural properties of a protein from its amino acid sequence
is a challenging yet important problem in biology. Structures are not known for
the vast majority of protein sequences, but structure is critical for
understanding function. Existing approaches for detecting structural similarity
between proteins from sequence are unable to recognize and exploit structural
patterns when sequences have diverged too far, limiting our ability to transfer
knowledge between structurally related proteins. We newly approach this problem
through the lens of representation learning. We introduce a framework that maps
any protein sequence to a sequence of vector embeddings --- one per amino acid
position --- that encode structural information. We train bidirectional long
short-term memory (LSTM) models on protein sequences with a two-part feedback
mechanism that incorporates information from (i) global structural similarity
between proteins and (ii) pairwise residue contact maps for individual
proteins. To enable learning from structural similarity information, we define
a novel similarity measure between arbitrary-length sequences of vector
embeddings based on a soft symmetric alignment (SSA) between them. Our method
is able to learn useful position-specific embeddings despite lacking direct
observations of position-level correspondence between sequences. We show
empirically that our multi-task framework outperforms other sequence-based
methods and even a top-performing structure-based alignment method when
predicting structural similarity, our goal. Finally, we demonstrate that our
learned embeddings can be transferred to other protein sequence problems,
improving the state-of-the-art in transmembrane domain prediction.
