  Recipe personalization through ingredient substitution has the potential to
help people meet their dietary needs and preferences, avoid potential
allergens, and ease culinary exploration in everyone's kitchen. To address
ingredient substitution, we build a benchmark, composed of a dataset of
substitution pairs with standardized splits, evaluation metrics, and baselines.
We further introduce Graph-based Ingredient Substitution Module (GISMo), a
novel model that leverages the context of a recipe as well as generic
ingredient relational information encoded within a graph to rank plausible
substitutions. We show through comprehensive experimental validation that GISMo
surpasses the best performing baseline by a large margin in terms of mean
reciprocal rank. Finally, we highlight the benefits of GISMo by integrating it
in an improved image-to-recipe generation pipeline, enabling recipe
personalization through user intervention. Quantitative and qualitative results
show the efficacy of our proposed system, paving the road towards truly
personalized cooking and tasting experiences.
