  This paper presents a 3-step system that estimates the real-time energy
expenditure of an individual in a non-intrusive way. First, using the user's
smart-phone's sensors, we build a Decision Tree model to recognize his physical
activity (\textit{running}, \textit{standing}, ...). Then, we use the detected
physical activity, the time and the user's speed to infer his daily activity
(\textit{watching TV}, \textit{going to the bathroom}, ...) through the use of
a reinforcement learning environment, the Partially Observable Markov Decision
Process framework. Once the daily activities are recognized, we translate this
information into energy expenditure using the compendium of physical
activities. By successfully detecting 8 physical activities at 90\%, we reached
an overall accuracy of 80\% in recognizing 17 different daily activities. This
result leads us to estimate the energy expenditure of the user with a mean
error of 26\% of the expected estimation.
