  Food classification is critical to the analysis of nutrients comprising foods
reported in dietary assessment. Advances in mobile and wearable sensors,
combined with new image based methods, particularly deep learning based
approaches, have shown great promise to improve the accuracy of food
classification to assess dietary intake. However, these approaches are
data-hungry and their performances are heavily reliant on the quantity and
quality of the available datasets for training the food classification model.
Existing food image datasets are not suitable for fine-grained food
classification and the following nutrition analysis as they lack fine-grained
and transparently derived food group based identification which are often
provided by trained dietitians with expert domain knowledge. In this paper, we
propose a framework to create a nutrition and food group based image database
that contains both visual and hierarchical food categorization information to
enhance links to the nutrient profile of each food. We design a protocol for
linking food group based food codes in the U.S. Department of Agriculture's
(USDA) Food and Nutrient Database for Dietary Studies (FNDDS) to a food image
dataset, and implement a web-based annotation tool for efficient deployment of
this protocol.Our proposed method is used to build a nutrition and food group
based image database including 16,114 food images representing the 74 most
frequently consumed What We Eat in America (WWEIA) food sub-categories in the
United States with 1,865 USDA food code matched to a nutrient database, the
USDA FNDDS nutrient database.
