  Instruction Fine-Tuning (IFT) significantly enhances the zero-shot
capabilities of pretrained Large Language Models (LLMs). While coding data is
known to boost reasoning abilities during LLM pretraining, its role in
activating internal reasoning capacities during IFT remains understudied. This
paper investigates a key question: How does coding data impact LLMs' reasoning
capacities during the IFT stage? To explore this, we thoroughly examine the
impact of coding data across different coding data proportions, model families,
sizes, and reasoning domains, from various perspectives. Specifically, we
create three IFT datasets with increasing coding data proportions, fine-tune
six LLM backbones across different families and scales on these datasets,
evaluate the tuned models' performance across twelve tasks in three reasoning
domains, and analyze the outcomes from three broad-to-granular perspectives:
overall, domain-level, and task-specific. Our holistic analysis provides
valuable insights in each perspective. First, coding data tuning enhances the
overall reasoning capabilities of LLMs across different model families and
scales. Moreover, the effect of coding data varies among different domains but
shows consistent trends across model families and scales within each domain.
Additionally, coding data generally yields comparable task-specific benefits
across different model families, with the optimal coding data proportions in
IFT datasets being task-specific.
