  Instruction tuning significantly enhances the performance of large language
models (LLMs) across various tasks. However, the procedure to optimizing the
mixing of instruction datasets for LLM fine-tuning is still poorly understood.
This study categorizes instructions into three primary types: NLP downstream
tasks, coding, and general chat. We explore the effects of instruction tuning
on different combinations of datasets on LLM performance, and find that certain
instruction types are more advantageous for specific applications but can
negatively impact other areas. This work provides insights into instruction
mixtures, laying the foundations for future research.
