Combining Deep Learning and Qualitative Spatial Reasoning to Learn
  Complex Structures from Sparse Examples with Noise
  Many modern machine learning approaches require vast amounts of training data
to learn new concepts; conversely, human learning often requires few
examples--sometimes only one--from which the learner can abstract structural
concepts. We present a novel approach to introducing new spatial structures to
an AI agent, combining deep learning over qualitative spatial relations with
various heuristic search algorithms. The agent extracts spatial relations from
a sparse set of noisy examples of block-based structures, and trains
convolutional and sequential models of those relation sets. To create novel
examples of similar structures, the agent begins placing blocks on a virtual
table, uses a CNN to predict the most similar complete example structure after
each placement, an LSTM to predict the most likely set of remaining moves
needed to complete it, and recommends one using heuristic search. We verify
that the agent learned the concept by observing its virtual block-building
activities, wherein it ranks each potential subsequent action toward building
its learned concept. We empirically assess this approach with human
participants' ratings of the block structures. Initial results and qualitative
evaluations of structures generated by the trained agent show where it has
generalized concepts from the training data, which heuristics perform best
within the search space, and how we might improve learning and execution.
