  Neural networks have long been used to model human intelligence, capturing
elements of behavior and cognition, and their neural basis. Recent advancements
in deep learning have enabled neural network models to reach and even surpass
human levels of intelligence in many respects, yet unlike humans, their ability
to learn new tasks quickly remains a challenge. People can reason not only in
familiar domains, but can also rapidly learn to reason through novel problems
and situations, raising the question of how well modern neural network models
capture human intelligence and in which ways they diverge. In this work, we
explore this gap by investigating human adults' ability to learn an abstract
reasoning task based on Sudoku from a brief instructional tutorial with
explanatory feedback for incorrect responses using a narrow range of training
examples. We find that participants who master the task do so within a small
number of trials and generalize well to puzzles outside of the training range.
We also find that most of those who master the task can describe a valid
solution strategy, and such participants perform better on transfer puzzles
than those whose strategy descriptions are vague or incomplete. Interestingly,
fewer than half of our human participants were successful in acquiring a valid
solution strategy, and this ability is associated with high school mathematics
education. We consider the challenges these findings pose for building
computational models that capture all aspects of our findings and point toward
a possible role for learning to engage in explanation-based reasoning to
support rapid learning and generalization.
