  Psychological curiosity plays a significant role in human intelligence to
enhance learning through exploration and information acquisition. In the
Artificial Intelligence (AI) community, artificial curiosity provides a natural
intrinsic motivation for efficient learning as inspired by human cognitive
development; meanwhile, it can bridge the existing gap between AI research and
practical application scenarios, such as overfitting, poor generalization,
limited training samples, high computational cost, etc. As a result,
curiosity-driven learning (CDL) has become increasingly popular, where agents
are self-motivated to learn novel knowledge. In this paper, we first present a
comprehensive review on the psychological study of curiosity and summarize a
unified framework for quantifying curiosity as well as its arousal mechanism.
Based on the psychological principle, we further survey the literature of
existing CDL methods in the fields of Reinforcement Learning, Recommendation,
and Classification, where both advantages and disadvantages as well as future
work are discussed. As a result, this work provides fruitful insights for
future CDL research and yield possible directions for further improvement.
