  Curiosity for machine agents has been a focus of lively research activity.
The study of human and animal curiosity, particularly specific curiosity, has
unearthed several properties that would offer important benefits for machine
learners, but that have not yet been well-explored in machine intelligence. In
this work, we conduct a comprehensive, multidisciplinary survey of the field of
animal and machine curiosity. As a principal contribution of this work, we use
this survey as a foundation to introduce and define what we consider to be five
of the most important properties of specific curiosity: 1) directedness towards
inostensible referents, 2) cessation when satisfied, 3) voluntary exposure, 4)
transience, and 5) coherent long-term learning. As a second main contribution
of this work, we show how these properties may be implemented together in a
proof-of-concept reinforcement learning agent: we demonstrate how the
properties manifest in the behaviour of this agent in a simple non-episodic
grid-world environment that includes curiosity-inducing locations and induced
targets of curiosity. As we would hope, our example of a computational specific
curiosity agent exhibits short-term directed behaviour while updating long-term
preferences to adaptively seek out curiosity-inducing situations. This work,
therefore, presents a landmark synthesis and translation of specific curiosity
to the domain of machine learning and reinforcement learning and provides a
novel view into how specific curiosity operates and in the future might be
integrated into the behaviour of goal-seeking, decision-making computational
agents in complex environments.
