  Emotions are intimately tied to motivation and the adaptation of behavior,
and many animal species show evidence of emotions in their behavior. Therefore,
emotions must be related to powerful mechanisms that aid survival, and,
emotions must be evolutionary continuous phenomena. How and why did emotions
evolve in nature, how do events get emotionally appraised, how do emotions
relate to cognitive complexity, and, how do they impact behavior and learning?
In this article I propose that all emotions are manifestations of reward
processing, in particular Temporal Difference (TD) error assessment.
Reinforcement Learning (RL) is a powerful computational model for the learning
of goal oriented tasks by exploration and feedback. Evidence indicates that
RL-like processes exist in many animal species. Key in the processing of
feedback in RL is the notion of TD error, the assessment of how much better or
worse a situation just became, compared to what was previously expected (or,
the estimated gain or loss of utility - or well-being - resulting from new
evidence). I propose a TDRL Theory of Emotion and discuss its ramifications for
our understanding of emotions in humans, animals and machines, and present
psychological, neurobiological and computational evidence in its support.
