  Large language models (LLMs) show amazing proficiency and fluency in the use
of language. Does this mean that they have also acquired insightful linguistic
knowledge about the language, to an extent that they can serve as an "expert
linguistic annotator"? In this paper, we examine the successes and limitations
of the GPT-3, ChatGPT, and GPT-4 models in analysis of sentence meaning
structure, focusing on the Abstract Meaning Representation (AMR; Banarescu et
al. 2013) parsing formalism, which provides rich graphical representations of
sentence meaning structure while abstracting away from surface forms. We
compare models' analysis of this semantic structure across two settings: 1)
direct production of AMR parses based on zero- and few-shot prompts, and 2)
indirect partial reconstruction of AMR via metalinguistic natural language
queries (e.g., "Identify the primary event of this sentence, and the predicate
corresponding to that event."). Across these settings, we find that models can
reliably reproduce the basic format of AMR, and can often capture core event,
argument, and modifier structure -- however, model outputs are prone to
frequent and major errors, and holistic analysis of parse acceptability shows
that even with few-shot demonstrations, models have virtually 0% success in
producing fully accurate parses. Eliciting natural language responses produces
similar patterns of errors. Overall, our findings indicate that these models
out-of-the-box can capture aspects of semantic structure, but there remain key
limitations in their ability to support fully accurate semantic analyses or
parses.
