  Protein representation learning is critical in various tasks in biology, such
as drug design and protein structure or function prediction, which has
primarily benefited from protein language models and graph neural networks.
These models can capture intrinsic patterns from protein sequences and
structures through masking and task-related losses. However, the learned
protein representations are usually not well optimized, leading to performance
degradation due to limited data, difficulty adapting to new tasks, etc. To
address this, we propose a new \underline{d}eep \underline{m}anifold
\underline{t}ransformation approach for universal \underline{p}rotein
\underline{r}epresentation \underline{l}earning (DMTPRL). It employs manifold
learning strategies to improve the quality and adaptability of the learned
embeddings. Specifically, we apply a novel manifold learning loss during
training based on the graph inter-node similarity. Our proposed DMTPRL method
outperforms state-of-the-art baselines on diverse downstream tasks across
popular datasets. This validates our approach for learning universal and robust
protein representations. We promise to release the code after acceptance.
