  FM Synthesis is a well-known algorithm used to generate complex timbre from a
compact set of design primitives. Typically featuring a MIDI interface, it is
usually impractical to control it from an audio source. On the other hand,
Differentiable Digital Signal Processing (DDSP) has enabled nuanced audio
rendering by Deep Neural Networks (DNNs) that learn to control differentiable
synthesis layers from arbitrary sound inputs. The training process involves a
corpus of audio for supervision, and spectral reconstruction loss functions.
Such functions, while being great to match spectral amplitudes, present a lack
of pitch direction which can hinder the joint optimization of the parameters of
FM synthesizers. In this paper, we take steps towards enabling continuous
control of a well-established FM synthesis architecture from an audio input.
Firstly, we discuss a set of design constraints that ease spectral optimization
of a differentiable FM synthesizer via a standard reconstruction loss. Next, we
present Differentiable DX7 (DDX7), a lightweight architecture for neural FM
resynthesis of musical instrument sounds in terms of a compact set of
parameters. We train the model on instrument samples extracted from the URMP
dataset, and quantitatively demonstrate its comparable audio quality against
selected benchmarks.
