  We introduce EQ-Bench, a novel benchmark designed to evaluate aspects of
emotional intelligence in Large Language Models (LLMs). We assess the ability
of LLMs to understand complex emotions and social interactions by asking them
to predict the intensity of emotional states of characters in a dialogue. The
benchmark is able to discriminate effectively between a wide range of models.
We find that EQ-Bench correlates strongly with comprehensive multi-domain
benchmarks like MMLU (Hendrycks et al., 2020) (r=0.97), indicating that we may
be capturing similar aspects of broad intelligence. Our benchmark produces
highly repeatable results using a set of 60 English-language questions. We also
provide open-source code for an automated benchmarking pipeline at
https://github.com/EQ-bench/EQ-Bench and a leaderboard at https://eqbench.com
