  Under sparse extrinsic reward settings, reinforcement learning has remained
challenging, despite surging interests in this field. Previous attempts suggest
that intrinsic reward can alleviate the issue caused by sparsity. In this
article, we present a novel intrinsic reward that is inspired by human
learning, as humans evaluate curiosity by comparing current observations with
historical knowledge. Our method involves training a self-supervised prediction
model, saving snapshots of the model parameters, and using nuclear norm to
evaluate the temporal inconsistency between the predictions of different
snapshots as intrinsic rewards. We also propose a variational weighting
mechanism to assign weight to different snapshots in an adaptive manner. Our
experimental results on various benchmark environments demonstrate the efficacy
of our method, which outperforms other intrinsic reward-based methods without
additional training costs and with higher noise tolerance. This work has been
submitted to the IEEE for possible publication. Copyright may be transferred
without notice, after which this version may no longer be accessible.
