  Accurate nutrition estimation helps people make informed decisions about
their dietary choices and is crucial for preventing serious health issues. We
present NutriBench, the first publicly available natural language meal
description based nutrition benchmark. NutriBench consists of 5,000
human-verified meal descriptions with macro-nutrient labels, including
carbohydrates, proteins, fats, and calories. The data is divided into 15
subsets varying in complexity based on the number, servings, and popularity of
the food items in the meal and the specificity of serving size descriptions. We
conducted an extensive evaluation of seven popular and state-of-the-art Large
Language Models (LLMs), including GPT-3.5, Llama-3, and a medical
domain-specific model with standard, Chain-of-Thought and Retrieval-Augmented
Generation strategies on our benchmark for carbohydrate estimation. We also
conducted a human study involving expert and non-expert participants and found
that LLMs can provide more accurate and faster predictions over a range of
complex queries. We present a thorough analysis and comparison of different
LLMs, highlighting the opportunities and challenges of using LLMs for nutrition
estimation in real-life scenarios. Our benchmark is publicly available at:
https://mehak126.github.io/nutribench.html
