  Many real-world systems can be described by mathematical models that are
human-comprehensible, easy to analyze and help explain the system's behavior.
Symbolic regression is a method that can automatically generate such models
from data. Historically, symbolic regression has been predominantly realized by
genetic programming, a method that evolves populations of candidate solutions
that are subsequently modified by genetic operators crossover and mutation.
However, this approach suffers from several deficiencies: it does not scale
well with the number of variables and samples in the training data - models
tend to grow in size and complexity without an adequate accuracy gain, and it
is hard to fine-tune the model coefficients using just genetic operators.
Recently, neural networks have been applied to learn the whole analytic model,
i.e., its structure and the coefficients, using gradient-based optimization
algorithms. This paper proposes a novel neural network-based symbolic
regression method that constructs physically plausible models based on even
very small training data sets and prior knowledge about the system. The method
employs an adaptive weighting scheme to effectively deal with multiple loss
function terms and an epoch-wise learning process to reduce the chance of
getting stuck in poor local optima. Furthermore, we propose a parameter-free
method for choosing the model with the best interpolation and extrapolation
performance out of all the models generated throughout the whole learning
process. We experimentally evaluate the approach on four test systems: the
TurtleBot 2 mobile robot, the magnetic manipulation system, the equivalent
resistance of two resistors in parallel, and the longitudinal force of the
anti-lock braking system. The results clearly show the potential of the method
to find parsimonious models that comply with the prior knowledge provided.
