  Lifelong language learning aims to stream learning NLP tasks while retaining
knowledge of previous tasks. Previous works based on the language model and
following data-free constraint approaches have explored formatting all data as
"begin token (\textit{B}) + context (\textit{C}) + question (\textit{Q}) +
answer (\textit{A})" for different tasks. However, they still suffer from
catastrophic forgetting and are exacerbated when the previous task's pseudo
data is insufficient for the following reasons: (1) The model has difficulty
generating task-corresponding pseudo data, and (2) \textit{A} is prone to error
when \textit{A} and \textit{C} are separated by \textit{Q} because the
information of the \textit{C} is diminished before generating \textit{A}.
Therefore, we propose the Ask Question First and Replay Question (AQF-RQ),
including a novel data format "\textit{BQCA}" and a new training task to train
pseudo questions of previous tasks. Experimental results demonstrate that
AQF-RQ makes it easier for the model to generate more pseudo data that match
corresponding tasks, and is more robust to both sufficient and insufficient
pseudo-data when the task boundary is both clear and unclear. AQF-RQ can
achieve only 0.36\% lower performance than multi-task learning.
