  Nutrient-based meal recommendations have the potential to help individuals
prevent or manage conditions such as diabetes and obesity. However, learning
people's food preferences and making recommendations that simultaneously appeal
to their palate and satisfy nutritional expectations are challenging. Existing
approaches either only learn high-level preferences or require a prolonged
learning period. We propose Yum-me, a personalized nutrient-based meal
recommender system designed to meet individuals' nutritional expectations,
dietary restrictions, and fine-grained food preferences. Yum-me enables a
simple and accurate food preference profiling procedure via a visual quiz-based
user interface, and projects the learned profile into the domain of
nutritionally appropriate food options to find ones that will appeal to the
user. We present the design and implementation of Yum-me, and further describe
and evaluate two innovative contributions. The first contriution is an open
source state-of-the-art food image analysis model, named FoodDist. We
demonstrate FoodDist's superior performance through careful benchmarking and
discuss its applicability across a wide array of dietary applications. The
second contribution is a novel online learning framework that learns food
preference from item-wise and pairwise image comparisons. We evaluate the
framework in a field study of 227 anonymous users and demonstrate that it
outperforms other baselines by a significant margin. We further conducted an
end-to-end validation of the feasibility and effectiveness of Yum-me through a
60-person user study, in which Yum-me improves the recommendation acceptance
rate by 42.63%.
