  In this paper we show how nuisance parameter marginalized posteriors can be
inferred directly from simulations in a likelihood-free setting, without having
to jointly infer the higher-dimensional interesting and nuisance parameter
posterior first and marginalize a posteriori. The result is that for an
inference task with a given number of interesting parameters, the number of
simulations required to perform likelihood-free inference can be kept (roughly)
the same irrespective of the number of additional nuisances to be marginalized
over. To achieve this we introduce two extensions to the standard
likelihood-free inference set-up. Firstly we show how nuisance parameters can
be re-cast as latent variables and hence automatically marginalized over in the
likelihood-free framework. Secondly, we derive an asymptotically optimal
compression from $N$ data down to $n$ summaries -- one per interesting
parameter -- such that the Fisher information is (asymptotically) preserved,
but the summaries are insensitive (to leading order) to the nuisance
parameters. This means that the nuisance marginalized inference task involves
learning $n$ interesting parameters from $n$ "nuisance hardened" data
summaries, regardless of the presence or number of additional nuisance
parameters to be marginalized over. We validate our approach on two examples
from cosmology: supernovae and weak lensing data analyses with nuisance
parameterized systematics. For the supernova problem, high-fidelity posterior
inference of $\Omega_m$ and $w_0$ (marginalized over systematics) can be
obtained from just a few hundred data simulations. For the weak lensing
problem, six cosmological parameters can be inferred from $\mathcal{O}(10^3)$
simulations, irrespective of whether ten additional nuisance parameters are
included in the problem or not.
