  Audio processors whose parameters are modified periodically over time are
often referred as time-varying or modulation based audio effects. Most existing
methods for modeling these type of effect units are often optimized to a very
specific circuit and cannot be efficiently generalized to other time-varying
effects. Based on convolutional and recurrent neural networks, we propose a
deep learning architecture for generic black-box modeling of audio processors
with long-term memory. We explore the capabilities of deep neural networks to
learn such long temporal dependencies and we show the network modeling various
linear and nonlinear, time-varying and time-invariant audio effects. In order
to measure the performance of the model, we propose an objective metric based
on the psychoacoustics of modulation frequency perception. We also analyze what
the model is actually learning and how the given task is accomplished.
