  This work explores the capability of conversational chatbots powered by large
language models (LLMs), to understand and characterize predicate symmetry, a
cognitive linguistic function traditionally believed to be an inherent human
trait. Leveraging in-context learning (ICL), a paradigm shift enabling chatbots
to learn new tasks from prompts without re-training, we assess the symmetrical
reasoning of five chatbots: ChatGPT 4, Huggingface chat AI, Microsoft's Copilot
AI, LLaMA through Perplexity, and Gemini Advanced. Using the Symmetry Inference
Sentence (SIS) dataset by Tanchip et al. (2020), we compare chatbot responses
against human evaluations to gauge their understanding of predicate symmetry.
Experiment results reveal varied performance among chatbots, with some
approaching human-like reasoning capabilities. Gemini, for example, reaches a
correlation of 0.85 with human scores, while providing a sounding justification
for each symmetry evaluation. This study underscores the potential and
limitations of LLMs in mirroring complex cognitive processes as symmetrical
reasoning.
