  We provide a deepened study of autocorrelations in Neural Markov Chain Monte
Carlo (NMCMC) simulations, a version of the traditional Metropolis algorithm
which employs neural networks to provide independent proposals. We illustrate
our ideas using the two-dimensional Ising model. We discuss several estimates
of autocorrelation times in the context of NMCMC, some inspired by analytical
results derived for the Metropolized Independent Sampler (MIS). We check their
reliability by estimating them on a small system where analytical results can
also be obtained. Based on the analytical results for MIS we propose a new loss
function and study its impact on the autocorelation times. Although, this
function's performance is a bit inferior to the traditional Kullback-Leibler
divergence, it offers two training algorithms which in some situations may be
beneficial. By studying a small, $4 \times 4$, system we gain access to the
dynamics of the training process which we visualize using several observables.
Furthermore, we quantitatively investigate the impact of imposing global
discrete symmetries of the system in the neural network training process on the
autocorrelation times. Eventually, we propose a scheme which incorporates
partial heat-bath updates which considerably improves the quality of the
training. The impact of the above enhancements is discussed for a $16 \times
16$ spin system. The summary of our findings may serve as a guidance to the
implementation of Neural Markov Chain Monte Carlo simulations for more
complicated models.
