  It is expected that progress toward true artificial intelligence will be
achieved through the emergence of a system that integrates representation
learning and complex reasoning (LeCun et al. 2015). In response to this
prediction, research has been conducted on implementing the symbolic reasoning
of a von Neumann computer in an artificial neural network (Graves et al. 2016;
Graves et al. 2014; Reed et al. 2015). However, these studies have many
limitations in realizing neural-symbolic integration (Jaeger. 2016). Here, we
present a new learning paradigm: a learning solving procedure (LSP) that learns
the procedure for solving complex problems. This is not accomplished merely by
learning input-output data, but by learning algorithms through a solving
procedure that obtains the output as a sequence of tasks for a given input
problem. The LSP neural network system not only learns simple problems of
addition and multiplication, but also the algorithms of complicated problems,
such as complex arithmetic expression, sorting, and Hanoi Tower. To realize
this, the LSP neural network structure consists of a deep neural network and
long short-term memory, which are recursively combined. Through
experimentation, we demonstrate the efficiency and scalability of LSP and its
validity as a mechanism of complex reasoning.
