  Although large language models (LLMs) are impressive in solving various
tasks, they can quickly be outdated after deployment. Maintaining their
up-to-date status is a pressing concern in the current era. This paper provides
a comprehensive review of recent advances in aligning LLMs with the
ever-changing world knowledge without re-training from scratch. We categorize
research works systemically and provide in-depth comparisons and discussion. We
also discuss existing challenges and highlight future directions to facilitate
research in this field. We release the paper list at
https://github.com/hyintell/awesome-refreshing-llms
