  State of the art Symbolic Regression (SR) methods currently build specialized
models, while the application of Large Language Models (LLMs) remains largely
unexplored. In this work, we introduce the first comprehensive framework that
utilizes LLMs for the task of SR. We propose In-Context Symbolic Regression
(ICSR), an SR method which iteratively refines a functional form with an LLM
and determines its coefficients with an external optimizer. ICSR leverages
LLMs' strong mathematical prior both to propose an initial set of possible
functions given the observations and to refine them based on their errors. Our
findings reveal that LLMs are able to successfully find symbolic equations that
fit the given data, matching or outperforming the overall performance of the
best SR baselines on four popular benchmarks, while yielding simpler equations
with better out of distribution generalization.
