  Neural networks adapt very well to distributed and continuous
representations, but struggle to generalize from small amounts of data.
Symbolic systems commonly achieve data efficient generalization by exploiting
modularity to benefit from local and discrete features of a representation.
These features allow symbolic programs to be improved one module at a time and
to experience combinatorial growth in the values they can successfully process.
However, it is difficult to design a component that can be used to form
symbolic abstractions and which is adequately overparametrized to learn
arbitrary high-dimensional transformations. I present Graph-based Symbolically
Synthesized Neural Networks (G-SSNNs), a class of neural modules that operate
on representations modified with synthesized symbolic programs to include a
fixed set of local and discrete features. I demonstrate that the choice of
injected features within a G-SSNN module modulates the data efficiency and
generalization of baseline neural models, creating predictable patterns of both
heightened and curtailed generalization. By training G-SSNNs, we also derive
information about desirable semantics of symbolic programs without manual
engineering. This information is compact and amenable to abstraction, but can
also be flexibly recontextualized for other high-dimensional settings. In
future work, I will investigate data efficient generalization and the
transferability of learned symbolic representations in more complex G-SSNN
designs based on more complex classes of symbolic programs. Experimental code
and data are available at
https://github.com/shlomenu/symbolically_synthesized_networks .
