  Calorie and nutrition research has attained increased interest in recent
years. But, due to the complexity of the problem, literature in this area
focuses on a limited subset of ingredients or dish types and simple
convolutional neural networks or traditional machine learning. Simultaneously,
estimation of ingredient portions can help improve calorie estimation and meal
re-production from a given image. In this paper, given a single cooking image,
a pipeline for calorie estimation and meal re-production for different servings
of the meal is proposed. The pipeline contains two stages. In the first stage,
a set of ingredients associated with the meal in the given image are predicted.
In the second stage, given image features and ingredients, portions of the
ingredients and finally the total meal calorie are simultaneously estimated
using a deep transformer-based model. Portion estimation introduced in the
model helps improve calorie estimation and is also beneficial for meal
re-production in different serving sizes. To demonstrate the benefits of the
pipeline, the model can be used for meal kits generation. To evaluate the
pipeline, the large scale dataset Recipe1M is used. Prior to experiments, the
Recipe1M dataset is parsed and explicitly annotated with portions of
ingredients. Experiments show that using ingredients and their portions
significantly improves calorie estimation. Also, a visual interface is created
in which a user can interact with the pipeline to reach accurate calorie
estimations and generate a meal kit for cooking purposes.
