  We study the in-context learning dynamics of large language models (LLMs)
using three instrumental learning tasks adapted from cognitive psychology. We
find that LLMs update their beliefs in an asymmetric manner and learn more from
better-than-expected outcomes than from worse-than-expected ones. Furthermore,
we show that this effect reverses when learning about counterfactual feedback
and disappears when no agency is implied. We corroborate these findings by
investigating idealized in-context learning agents derived through
meta-reinforcement learning, where we observe similar patterns. Taken together,
our results contribute to our understanding of how in-context learning works by
highlighting that the framing of a problem significantly influences how
learning occurs, a phenomenon also observed in human cognition.
