  Building autonomous agents able to grow open-ended repertoires of skills
across their lives is a fundamental goal of artificial intelligence (AI). A
promising developmental approach recommends the design of intrinsically
motivated agents that learn new skills by generating and pursuing their own
goals - autotelic agents. But despite recent progress, existing algorithms
still show serious limitations in terms of goal diversity, exploration,
generalisation or skill composition. This perspective calls for the immersion
of autotelic agents into rich socio-cultural worlds, an immensely important
attribute of our environment that shapes human cognition but is mostly omitted
in modern AI. Inspired by the seminal work of Vygotsky, we propose Vygotskian
autotelic agents - agents able to internalise their interactions with others
and turn them into cognitive tools. We focus on language and show how its
structure and informational content may support the development of new
cognitive functions in artificial agents as it does in humans. We justify the
approach by uncovering several examples of new artificial cognitive functions
emerging from interactions between language and embodiment in recent works at
the intersection of deep reinforcement learning and natural language
processing. Looking forward, we highlight future opportunities and challenges
for Vygotskian Autotelic AI research, including the use of language models as
cultural models supporting artificial cognitive development.
