  Individuality is essential in human society, which induces the division of
labor and thus improves the efficiency and productivity. Similarly, it should
also be the key to multi-agent cooperation. Inspired by that individuality is
of being an individual separate from others, we propose a simple yet efficient
method for the emergence of individuality (EOI) in multi-agent reinforcement
learning (MARL). EOI learns a probabilistic classifier that predicts a
probability distribution over agents given their observation and gives each
agent an intrinsic reward of being correctly predicted by the classifier. The
intrinsic reward encourages the agents to visit their own familiar
observations, and learning the classifier by such observations makes the
intrinsic reward signals stronger and the agents more identifiable. To further
enhance the intrinsic reward and promote the emergence of individuality, two
regularizers are proposed to increase the discriminability of the classifier.
We implement EOI on top of popular MARL algorithms. Empirically, we show that
EOI significantly outperforms existing methods in a variety of multi-agent
cooperative scenarios.
