  Lifelong learning (LL) is an important ability for NLP models to learn new
tasks continuously. Architecture-based approaches are reported to be effective
implementations for LL models. However, it is non-trivial to extend previous
approaches to domain incremental LL scenarios since they either require access
to task identities in the testing phase or cannot handle samples from unseen
tasks. In this paper, we propose \textbf{Diana}: a
\underline{d}ynam\underline{i}c \underline{a}rchitecture-based
lifelo\underline{n}g le\underline{a}rning model that tries to learn a sequence
of tasks with a prompt-enhanced language model. Four types of hierarchically
organized prompts are used in Diana to capture knowledge from different
granularities. Specifically, we dedicate task-level prompts to capture
task-specific knowledge to retain high LL performances and maintain
instance-level prompts to learn knowledge shared across input samples to
improve the model's generalization performance. Moreover, we dedicate separate
prompts to explicitly model unseen tasks and introduce a set of prompt key
vectors to facilitate knowledge sharing between tasks. Extensive experiments
demonstrate that Diana outperforms state-of-the-art LL models, especially in
handling unseen tasks. We release the code and data at
\url{https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/diana}.
