  Striking progress has recently been made in understanding human cognition by
analyzing how its neuronal underpinnings are engaged in different modes of
information processing. Specifically, neural information can be decomposed into
synergistic, redundant, and unique features, with synergistic components being
particularly aligned with complex cognition. However, two fundamental questions
remain unanswered: (a) precisely how and why a cognitive system can become
highly synergistic; and (b) how these informational states map onto artificial
neural networks in various learning modes. To address these questions, here we
employ an information-decomposition framework to investigate the information
processing strategies adopted by simple artificial neural networks performing a
variety of cognitive tasks in both supervised and reinforcement learning
settings. Our results show that synergy increases as neural networks learn
multiple diverse tasks. Furthermore, performance in tasks requiring integration
of multiple information sources critically relies on synergistic neurons.
Finally, randomly turning off neurons during training through dropout increases
network redundancy, corresponding to an increase in robustness. Overall, our
results suggest that while redundant information is required for robustness to
perturbations in the learning process, synergistic information is used to
combine information from multiple modalities -- and more generally for flexible
and efficient learning. These findings open the door to new ways of
investigating how and why learning systems employ specific
information-processing strategies, and support the principle that the capacity
for general-purpose learning critically relies in the system's information
dynamics.
