  Evolution gave rise to human and animal intelligence here on Earth. We argue
that the path to developing artificial human-like-intelligence will pass
through mimicking the evolutionary process in a nature-like simulation. In
Nature, there are two processes driving the development of the brain: evolution
and learning. Evolution acts slowly, across generations, and amongst other
things, it defines what agents learn by changing their internal reward
function. Learning acts fast, across one's lifetime, and it quickly updates
agents' policy to maximise pleasure and minimise pain. The reward function is
slowly aligned with the fitness function by evolution, however, as agents
evolve the environment and its fitness function also change, increasing the
misalignment between reward and fitness. It is extremely computationally
expensive to replicate these two processes in simulation. This work proposes
Evolution via Evolutionary Reward (EvER) that allows learning to
single-handedly drive the search for policies with increasingly evolutionary
fitness by ensuring the alignment of the reward function with the fitness
function. In this search, EvER makes use of the whole state-action trajectories
that agents go through their lifetime. In contrast, current evolutionary
algorithms discard this information and consequently limit their potential
efficiency at tackling sequential decision problems. We test our algorithm in
two simple bio-inspired environments and show its superiority at generating
more capable agents at surviving and reproducing their genes when compared with
a state-of-the-art evolutionary algorithm.
