  We introduce an end-to-end deep learning architecture called the wide-band
butterfly network (WideBNet) for approximating the inverse scattering map from
wide-band scattering data. This architecture incorporates tools from
computational harmonic analysis, such as the butterfly factorization, and
traditional multi-scale methods, such as the Cooley-Tukey FFT algorithm, to
drastically reduce the number of trainable parameters to match the inherent
complexity of the problem. As a result WideBNet is efficient: it requires fewer
training points than off-the-shelf architectures, and has stable training
dynamics, thus it can rely on standard weight initialization strategies. The
architecture automatically adapts to the dimensions of the data with only a few
hyper-parameters that the user must specify. WideBNet is able to produce images
that are competitive with optimization-based approaches, but at a fraction of
the cost, and we also demonstrate numerically that it learns to super-resolve
scatterers in the full aperture scattering setup.
