  We present a new framework for vision-based estimation of calorific
expenditure from RGB-D data - the first that is validated on physical gas
exchange measurements and applied to daily living scenarios. Deriving a
person's energy expenditure from sensors is an important tool in tracking
physical activity levels for health and lifestyle monitoring. Most existing
methods use metabolic lookup tables (METs) for a manual estimate or systems
with inertial sensors which ultimately require users to wear devices. In
contrast, the proposed pose-invariant and individual-independent vision
framework allows for a remote estimation of calorific expenditure. We
introduce, and evaluate our approach on, a new dataset called SPHERE-calorie,
for which visual estimates can be compared against simultaneously obtained,
indirect calorimetry measures based on gas exchange. % based on per breath gas
exchange. We conclude from our experiments that the proposed vision pipeline is
suitable for home monitoring in a controlled environment, with calorific
expenditure estimates above accuracy levels of commonly used manual estimations
via METs. With the dataset released, our work establishes a baseline for future
research for this little-explored area of computer vision.
