  Deep reinforcement learning can match or exceed human performance in stable
contexts, but with minor changes to the environment artificial networks, unlike
humans, often cannot adapt. Humans rely on a combination of heuristics to
simplify computational load and imagination to extend experiential learning to
new and more challenging environments. Motivated by theories of the
hierarchical organization of the human prefrontal networks, we have developed a
model of hierarchical reinforcement learning that combines both heuristics and
imagination into a stumbler-strategist network. We test performance of this
network using Wythoff's game, a gridworld environment with a known optimal
strategy. We show that a heuristic labeling of each position as hot or cold,
combined with imagined play, both accelerates learning and promotes transfer to
novel games, while also improving model interpretability.
