  The ability of Large Language Models (LLMs) to encode syntactic and semantic
structures of language is well examined in NLP. Additionally, analogy
identification, in the form of word analogies are extensively studied in the
last decade of language modeling literature. In this work we specifically look
at how LLMs' abilities to capture sentence analogies (sentences that convey
analogous meaning to each other) vary with LLMs' abilities to encode syntactic
and semantic structures of sentences. Through our analysis, we find that LLMs'
ability to identify sentence analogies is positively correlated with their
ability to encode syntactic and semantic structures of sentences. Specifically,
we find that the LLMs which capture syntactic structures better, also have
higher abilities in identifying sentence analogies.
