  Symbolic regression (SR) poses a significant challenge for randomized search
heuristics due to its reliance on the synthesis of expressions for input-output
mappings. Although traditional genetic programming (GP) algorithms have
achieved success in various domains, they exhibit limited performance when
tree-based representations are used for SR. To address these limitations, we
introduce a novel SR approach called Fourier Tree Growing (FTG) that draws
insights from functional analysis. This new perspective enables us to perform
optimization directly in a different space, thus avoiding intricate symbolic
expressions. Our proposed algorithm exhibits significant performance
improvements over traditional GP methods on a range of classical
one-dimensional benchmarking problems. To identify and explain limiting factors
of GP and FTG, we perform experiments on a large-scale polynomials benchmark
with high-order polynomials up to degree 100. To the best of the authors'
knowledge, this work represents the pioneering application of functional
analysis in addressing SR problems. The superior performance of the proposed
algorithm and insights into the limitations of GP open the way for further
advancing GP for SR and related areas of explainable machine learning.
