  The integration of Large Language Models (LLM) with Knowledge Representation
Learning (KRL) signifies a significant advancement in the field of artificial
intelligence (AI), enhancing the ability to capture and utilize both structure
and textual information. Despite the increasing research on enhancing KRL with
LLMs, a thorough survey that analyse processes of these enhanced models is
conspicuously absent. Our survey addresses this by categorizing these models
based on three distinct Transformer architectures, and by analyzing
experimental data from various KRL downstream tasks to evaluate the strengths
and weaknesses of each approach. Finally, we identify and explore potential
future research directions in this emerging yet underexplored domain.
