  In this paper, we present a new task that investigates how people interact
with and make judgments about towers of blocks. In Experiment~1, participants
in the lab solved a series of problems in which they had to re-configure three
blocks from an initial to a final configuration. We recorded whether they used
one hand or two hands to do so. In Experiment~2, we asked participants online
to judge whether they think the person in the lab used one or two hands. The
results revealed a close correspondence between participants' actions in the
lab, and the mental simulations of participants online. To explain
participants' actions and mental simulations, we develop a model that plans
over a symbolic representation of the situation, executes the plan using a
geometric solver, and checks the plan's feasibility by taking into account the
physical constraints of the scene. Our model explains participants' actions and
judgments to a high degree of quantitative accuracy.
