  We introduce a framework that predicts the goals behind observable human
action in video. Motivated by evidence in developmental psychology, we leverage
video of unintentional action to learn video representations of goals without
direct supervision. Our approach models videos as contextual trajectories that
represent both low-level motion and high-level action features. Experiments and
visualizations show our trained model is able to predict the underlying goals
in video of unintentional action. We also propose a method to "automatically
correct" unintentional action by leveraging gradient signals of our model to
adjust latent trajectories. Although the model is trained with minimal
supervision, it is competitive with or outperforms baselines trained on large
(supervised) datasets of successfully executed goals, showing that observing
unintentional action is crucial to learning about goals in video. Project page:
https://aha.cs.columbia.edu/
