  Researchers are using deep learning models to explore the emergence of
language in various language games, where agents interact and develop an
emergent language to solve tasks. We focus on the factors that determine the
expressivity of emergent languages, which reflects the amount of information
about input spaces those languages are capable of encoding. We measure the
expressivity of emergent languages based on the generalisation performance
across different games, and demonstrate that the expressivity of emergent
languages is a trade-off between the complexity and unpredictability of the
context those languages emerged from. Another contribution of this work is the
discovery of message type collapse, i.e. the number of unique messages is lower
than that of inputs. We also show that using the contrastive loss proposed by
Chen et al. (2020) can alleviate this problem.
