  Advanced biological intelligence learns efficiently from an information-rich
stream of stimulus information, even when feedback on behaviour quality is
sparse or absent. Such learning exploits implicit assumptions about task
domains. We refer to such learning as Domain-Adapted Learning (DAL). In
contrast, AI learning algorithms rely on explicit externally provided measures
of behaviour quality to acquire fit behaviour. This imposes an information
bottleneck that precludes learning from diverse non-reward stimulus
information, limiting learning efficiency. We consider the question of how
biological evolution circumvents this bottleneck to produce DAL. We propose
that species first evolve the ability to learn from reward signals, providing
inefficient (bottlenecked) but broad adaptivity. From there, integration of
non-reward information into the learning process can proceed via gradual
accumulation of biases induced by such information on specific task domains.
This scenario provides a biologically plausible pathway towards
bottleneck-free, domain-adapted learning. Focusing on the second phase of this
scenario, we set up a population of NNs with reward-driven learning modelled as
Reinforcement Learning (A2C), and allow evolution to improve learning
efficiency by integrating non-reward information into the learning process
using a neuromodulatory update mechanism. On a navigation task in continuous 2D
space, evolved DAL agents show a 300-fold increase in learning speed compared
to pure RL agents. Evolution is found to eliminate reliance on reward
information altogether, allowing DAL agents to learn from non-reward
information exclusively, using local neuromodulation-based connection weight
updates only. Code available at github.com/aislab/dal.
