  We introduce a set of gradient-flow-guided adaptive importance sampling (IS)
transformations to stabilize Monte-Carlo approximations of point-wise leave one
out cross-validated (LOO) predictions for Bayesian classification models. One
can leverage this methodology for assessing model generalizability by for
instance computing a LOO analogue to the AIC or computing LOO ROC/PRC curves
and derived metrics like the AUROC and AUPRC. By the calculus of variations and
gradient flow, we derive two simple nonlinear single-step transformations that
utilize gradient information to shift a model's pre-trained full-data posterior
closer to the target LOO posterior predictive distributions. In doing so, the
transformations stabilize importance weights. Because the transformations
involve the gradient of the likelihood function, the resulting Monte Carlo
integral depends on Jacobian determinants with respect to the model Hessian. We
derive closed-form exact formulae for these Jacobian determinants in the cases
of logistic regression and shallow ReLU-activated artificial neural networks,
and provide a simple approximation that sidesteps the need to compute full
Hessian matrices and their spectra. We test the methodology on an $n\ll p$
dataset that is known to produce unstable LOO IS weights.
