  With the proliferation of large language models (LLMs), the comprehensive
alignment of such models across multiple tasks has emerged as a critical area
of research. Existing alignment methodologies primarily address single task,
such as multi-turn dialogue, coding, mathematical problem-solving, and tool
usage. However, AI-driven products that leverage language models usually
necessitate a fusion of these abilities to function effectively in real-world
scenarios. Moreover, the considerable computational resources required for
proper alignment of LLMs underscore the need for a more robust, efficient, and
encompassing approach to multi-task alignment, ensuring improved generative
performance. In response to these challenges, we introduce a novel technique
termed Mixture-of-Instructions (MoI), which employs a strategy of instruction
concatenation combined with diverse system prompts to boost the alignment
efficiency of language models. We have also compiled a diverse set of seven
benchmark datasets to rigorously evaluate the alignment efficacy of the
MoI-enhanced language model. Our methodology was applied to the open-source
Qwen-7B-chat model, culminating in the development of Qwen-SFT-MoI. This
enhanced model demonstrates significant advancements in generative capabilities
across coding, mathematics, and tool use tasks.
