  Infants explore their complex physical and social environment in an organized
way. To gain insight into what intrinsic motivations may help structure this
exploration, we create a virtual infant agent and place it in a
developmentally-inspired 3D environment with no external rewards. The
environment has a virtual caregiver agent with the capability to interact
contingently with the infant agent in ways that resemble play. We test
intrinsic reward functions that are similar to motivations that have been
proposed to drive exploration in humans: surprise, uncertainty, novelty, and
learning progress. These generic reward functions lead the infant agent to
explore its environment and discover the contingencies that are embedded into
the caregiver agent. The reward functions that are proxies for novelty and
uncertainty are the most successful in generating diverse experiences and
activating the environment contingencies. We also find that learning a world
model in the presence of an attentive caregiver helps the infant agent learn
how to predict scenarios with challenging social and physical dynamics. Taken
together, our findings provide insight into how curiosity-like intrinsic
rewards and contingent social interaction lead to dynamic social behavior and
the creation of a robust predictive world model.
