  We investigate the use of approximate Bayesian neural networks (BNNs) in
modeling hundreds of time-delay gravitational lenses for Hubble constant
($H_0$) determination. Our BNN was trained on synthetic HST-quality images of
strongly lensed active galactic nuclei (AGN) with lens galaxy light included.
The BNN can accurately characterize the posterior PDFs of model parameters
governing the elliptical power-law mass profile in an external shear field. We
then propagate the BNN-inferred posterior PDFs into ensemble $H_0$ inference,
using simulated time delay measurements from a plausible dedicated monitoring
campaign. Assuming well-measured time delays and a reasonable set of priors on
the environment of the lens, we achieve a median precision of $9.3$\% per lens
in the inferred $H_0$. A simple combination of 200 test-set lenses results in a
precision of 0.5 $\textrm{km s}^{-1} \textrm{ Mpc}^{-1}$ ($0.7\%$), with no
detectable bias in this $H_0$ recovery test. The computation time for the
entire pipeline -- including the training set generation, BNN training, and
$H_0$ inference -- translates to 9 minutes per lens on average for 200 lenses
and converges to 6 minutes per lens as the sample size is increased. Being
fully automated and efficient, our pipeline is a promising tool for exploring
ensemble-level systematics in lens modeling for $H_0$ inference.
