  In the field of equation learning, exhaustively considering all possible
equations derived from a basis function dictionary is infeasible. Sparse
regression and greedy algorithms have emerged as popular approaches to tackle
this challenge. However, the presence of multicollinearity poses difficulties
for sparse regression techniques, and greedy steps may inadvertently exclude
terms of the true equation, leading to reduced identification accuracy. In this
article, we present an approach that strikes a balance between
comprehensiveness and efficiency in equation learning. Inspired by stepwise
regression, our approach combines the coefficient of determination, $R^2$, and
the Bayesian model evidence, $p(\boldsymbol y|\mathcal M)$, in a novel way. Our
procedure is characterized by a comprehensive search with just a minor
reduction of the model space at each iteration step. With two flavors of our
approach and the adoption of $p(\boldsymbol y|\mathcal M)$ for bi-directional
stepwise regression, we present a total of three new avenues for equation
learning. Through three extensive numerical experiments involving random
polynomials and dynamical systems, we compare our approach against four
state-of-the-art methods and two standard approaches. The results demonstrate
that our comprehensive search approach surpasses all other methods in terms of
identification accuracy. In particular, the second flavor of our approach
establishes an efficient overfitting penalty solely based on $R^2$, which
achieves highest rates of exact equation recovery.
