  Symbolic regression (SR) is the problem of learning a symbolic expression
from numerical data. Recently, deep neural models trained on
procedurally-generated synthetic datasets showed competitive performance
compared to more classical Genetic Programming (GP) algorithms. Unlike their GP
counterparts, these neural approaches are trained to generate expressions from
datasets given as context. This allows them to produce accurate expressions in
a single forward pass at test time. However, they usually do not benefit from
search abilities, which result in low performance compared to GP on
out-of-distribution datasets. In this paper, we propose a novel method which
provides the best of both worlds, based on a Monte-Carlo Tree Search procedure
using a context-aware neural mutation model, which is initially pre-trained to
learn promising mutations, and further refined from successful experiences in
an online fashion. The approach demonstrates state-of-the-art performance on
the well-known \texttt{SRBench} benchmark.
