  This paper explores an intriguing observation: fine-tuning a large language
model (LLM) with responses generated by a LLM often yields better results than
using responses generated by humans. We conduct an in-depth investigation to
understand why this occurs. Contrary to the common belief that these instances
is simply due to the more detailed nature of LLM-generated content, our study
identifies another contributing factor: an LLM is inherently more "familiar"
with LLM generated responses. This familiarity is evidenced by lower perplexity
before fine-tuning. We design a series of experiments to understand the impact
of the "familiarity" and our conclusion reveals that this "familiarity"
significantly impacts learning performance. Training with LLM-generated
responses not only enhances performance but also helps maintain the model's
capabilities in other tasks after fine-tuning on a specific task.
