  When performing complex tasks, humans naturally reason at multiple temporal
and spatial resolutions simultaneously. We contend that for an artificially
intelligent agent to effectively model human teammates, i.e., demonstrate
computational theory of mind (ToM), it should do the same. In this paper, we
present an approach for integrating high and low-resolution spatial and
temporal information to predict human behavior in real time and evaluate it on
data collected from human subjects performing simulated urban search and rescue
(USAR) missions in a Minecraft-based environment. Our model composes neural
networks for high and low-resolution feature extraction with a neural network
for behavior prediction, with all three networks trained simultaneously. The
high-resolution extractor encodes dynamically changing goals robustly by taking
as input the Manhattan distance difference between the humans' Minecraft
avatars and candidate goals in the environment for the latest few actions,
computed from a high-resolution gridworld representation. In contrast, the
low-resolution extractor encodes participants' historical behavior using a
historical state matrix computed from a low-resolution graph representation.
Through supervised learning, our model acquires a robust prior for human
behavior prediction, and can effectively deal with long-term observations. Our
experimental results demonstrate that our method significantly improves
prediction accuracy compared to approaches that only use high-resolution
information.
