  We present Subtractive Training, a simple and novel method for synthesizing
individual musical instrument stems given other instruments as context. This
method pairs a dataset of complete music mixes with 1) a variant of the dataset
lacking a specific stem, and 2) LLM-generated instructions describing how the
missing stem should be reintroduced. We then fine-tune a pretrained
text-to-audio diffusion model to generate the missing instrument stem, guided
by both the existing stems and the text instruction. Our results demonstrate
Subtractive Training's efficacy in creating authentic drum stems that
seamlessly blend with the existing tracks. We also show that we can use the
text instruction to control the generation of the inserted stem in terms of
rhythm, dynamics, and genre, allowing us to modify the style of a single
instrument in a full song while keeping the remaining instruments the same.
Lastly, we extend this technique to MIDI formats, successfully generating
compatible bass, drum, and guitar parts for incomplete arrangements.
