  Reinforcement learning can learn amortised design policies for designing
sequences of experiments. However, current amortised methods rely on estimators
of expected information gain (EIG) that require an exponential number of
samples on the magnitude of the EIG to achieve an unbiased estimation. We
propose the use of an alternative estimator based on the cross-entropy of the
joint model distribution and a flexible proposal distribution. This proposal
distribution approximates the true posterior of the model parameters given the
experimental history and the design policy. Our method overcomes the
exponential-sample complexity of previous approaches and provide more accurate
estimates of high EIG values. More importantly, it allows learning of superior
design policies, and is compatible with continuous and discrete design spaces,
non-differentiable likelihoods and even implicit probabilistic models.
