  To overcome the limitations of Neural Programmer-Interpreters (NPI) in its
universality and learnability, we propose the incorporation of combinator
abstraction into neural programing and a new NPI architecture to support this
abstraction, which we call Combinatory Neural Programmer-Interpreter (CNPI).
Combinator abstraction dramatically reduces the number and complexity of
programs that need to be interpreted by the core controller of CNPI, while
still allowing the CNPI to represent and interpret arbitrary complex programs
by the collaboration of the core with the other components. We propose a small
set of four combinators to capture the most pervasive programming patterns. Due
to the finiteness and simplicity of this combinator set and the offloading of
some burden of interpretation from the core, we are able construct a CNPI that
is universal with respect to the set of all combinatorizable programs, which is
adequate for solving most algorithmic tasks. Moreover, besides supervised
training on execution traces, CNPI can be trained by policy gradient
reinforcement learning with appropriately designed curricula.
