  The process of identifying changes or transformations in a scene along with
the ability of reasoning about their causes and effects, is a key aspect of
intelligence. In this work we go beyond recent advances in computational
perception, and introduce a more challenging task, Image-based Event-Sequencing
(IES). In IES, the task is to predict a sequence of actions required to
rearrange objects from the configuration in an input source image to the one in
the target image. IES also requires systems to possess inductive
generalizability. Motivated from evidence in cognitive development, we compile
the first IES dataset, the Blocksworld Image Reasoning Dataset (BIRD) which
contains images of wooden blocks in different configurations, and the sequence
of moves to rearrange one configuration to the other. We first explore the use
of existing deep learning architectures and show that these end-to-end methods
under-perform in inferring temporal event-sequences and fail at inductive
generalization. We then propose a modular two-step approach: Visual Perception
followed by Event-Sequencing, and demonstrate improved performance by combining
learning and reasoning. Finally, by showing an extension of our approach on
natural images, we seek to pave the way for future research on event sequencing
for real world scenes.
