  The reward system is one of the fundamental drivers of animal behaviors and
is critical for survival and reproduction. Despite its importance, the problem
of how the reward system has evolved is underexplored. In this paper, we try to
replicate the evolution of biologically plausible reward functions and
investigate how environmental conditions affect evolved rewards' shape. For
this purpose, we developed a population-based decentralized evolutionary
simulation framework, where agents maintain their energy level to live longer
and produce more children. Each agent inherits its reward function from its
parent subject to mutation and learns to get rewards via reinforcement learning
throughout its lifetime. Our results show that biologically reasonable positive
rewards for food acquisition and negative rewards for motor action can evolve
from randomly initialized ones. However, we also find that the rewards for
motor action diverge into two modes: largely positive and slightly negative.
The emergence of positive motor action rewards is surprising because it can
make agents too active and inefficient in foraging. In environments with poor
and poisonous foods, the evolution of rewards for less important foods tends to
be unstable, while rewards for normal foods are still stable. These results
demonstrate the usefulness of our simulation environment and energy-dependent
birth and death model for further studies of the origin of reward systems.
