  We present a framework for automatically structuring and training fast,
approximate, deep neural surrogates of stochastic simulators. Unlike
traditional approaches to surrogate modeling, our surrogates retain the
interpretable structure and control flow of the reference simulator. Our
surrogates target stochastic simulators where the number of random variables
itself can be stochastic and potentially unbounded. Our framework further
enables an automatic replacement of the reference simulator with the surrogate
when undertaking amortized inference. The fidelity and speed of our surrogates
allow for both faster stochastic simulation and accurate and substantially
faster posterior inference. Using an illustrative yet non-trivial example we
show our surrogates' ability to accurately model a probabilistic program with
an unbounded number of random variables. We then proceed with an example that
shows our surrogates are able to accurately model a complex structure like an
unbounded stack in a program synthesis example. We further demonstrate how our
surrogate modeling technique makes amortized inference in complex black-box
simulators an order of magnitude faster. Specifically, we do simulator-based
materials quality testing, inferring safety-critical latent internal
temperature profiles of composite materials undergoing curing.
