  This paper proposes a novel deep symbolic regression approach to enhance the
robustness and interpretability of data-driven mathematical expression
discovery. Despite the success of the state-of-the-art method, DSR, it is built
on recurrent neural networks, purely guided by data fitness, and potentially
meet tail barriers, which can zero out the policy gradient and cause
inefficient model updates. To overcome these limitations, we use transformers
in conjunction with breadth-first-search to improve the learning performance.
We use Bayesian information criterion (BIC) as the reward function to
explicitly account for the expression complexity and optimize the trade-off
between interpretability and data fitness. We propose a modified risk-seeking
policy that not only ensures the unbiasness of the gradient, but also removes
the tail barriers, thus ensuring effective updates from top performers. Through
a series of benchmarks and systematic experiments, we demonstrate the
advantages of our approach.
