  To overcome the computational bottleneck of various data perturbation
procedures such as the bootstrap and cross validations, we propose the
Generative Multiple-purpose Sampler (GMS), which constructs a generator
function to produce solutions of weighted M-estimators from a set of given
weights and tuning parameters. The GMS is implemented by a single optimization
without having to repeatedly evaluate the minimizers of weighted losses, and is
thus capable of significantly reducing the computational time. We demonstrate
that the GMS framework enables the implementation of various statistical
procedures that would be unfeasible in a conventional framework, such as the
iterated bootstrap, bootstrapped cross-validation for penalized likelihood,
bootstrapped empirical Bayes with nonparametric maximum likelihood, etc. To
construct a computationally efficient generator function, we also propose a
novel form of neural network called the \emph{weight multiplicative multilayer
perceptron} to achieve fast convergence. Our numerical results demonstrate that
the new neural network structure enjoys a few orders of magnitude speed
advantage in comparison to the conventional one. An R package called GMS is
provided, which runs under Pytorch to implement the proposed methods and allows
the user to provide a customized loss function to tailor to their own models of
interest.
