  We present a new multimodal dataset called Visual Recipe Flow, which enables
us to learn each cooking action result in a recipe text. The dataset consists
of object state changes and the workflow of the recipe text. The state change
is represented as an image pair, while the workflow is represented as a recipe
flow graph (r-FG). The image pairs are grounded in the r-FG, which provides the
cross-modal relation. With our dataset, one can try a range of applications,
from multimodal commonsense reasoning and procedural text generation.
