  Programming by Example (PBE) is the task of inducing computer programs from
input-output examples. It can be seen as a type of machine learning where the
hypothesis space is the set of legal programs in some programming language.
Recent work on differentiable interpreters relaxes the discrete space of
programs into a continuous space so that search over programs can be performed
using gradient-based optimization. While conceptually powerful, so far
differentiable interpreter-based program synthesis has only been capable of
solving very simple problems. In this work, we study modeling choices that
arise when constructing a differentiable programming language and their impact
on the success of synthesis. The main motivation for the modeling choices comes
from functional programming: we study the effect of memory allocation schemes,
immutable data, type systems, and built-in control-flow structures. Empirically
we show that incorporating functional programming ideas into differentiable
programming languages allows us to learn much more complex programs than is
possible with existing differentiable languages.
