  In this paper, we present an efficient neural network for end-to-end general
purpose audio source separation. Specifically, the backbone structure of this
convolutional network is the SUccessive DOwnsampling and Resampling of
Multi-Resolution Features (SuDoRMRF) as well as their aggregation which is
performed through simple one-dimensional convolutions. In this way, we are able
to obtain high quality audio source separation with limited number of floating
point operations, memory requirements, number of parameters and latency. Our
experiments on both speech and environmental sound separation datasets show
that SuDoRMRF performs comparably and even surpasses various state-of-the-art
approaches with significantly higher computational resource requirements.
