  We consider Bayesian inverse problems wherein the unknown state is assumed to
be a function with discontinuous structure a priori. A class of prior
distributions based on the output of neural networks with heavy-tailed weights
is introduced, motivated by existing results concerning the infinite-width
limit of such networks. We show theoretically that samples from such priors
have desirable discontinuous-like properties even when the network width is
finite, making them appropriate for edge-preserving inversion. Numerically we
consider deconvolution problems defined on one- and two-dimensional spatial
domains to illustrate the effectiveness of these priors; MAP estimation,
dimension-robust MCMC sampling and ensemble-based approximations are utilized
to probe the posterior distribution. The accuracy of point estimates is shown
to exceed those obtained from non-heavy tailed priors, and uncertainty
estimates are shown to provide more useful qualitative information.
