  When applied to the non-linear matter distribution of the universe, neural
networks have been shown to be very statistically sensitive probes of
cosmological parameters, such as the linear perturbation amplitude $\sigma_8$.
However, when used as a "black box", neural networks are not robust to baryonic
uncertainty. We propose a robust architecture for constraining primordial
non-Gaussianity $f_{NL}$, by training a neural network to locally estimate
$\sigma_8$, and correlating these local estimates with the large-scale density
field. We apply our method to N-body simulations, and show that
$\sigma(f_{NL})$ is 3.5 times better than the constraint obtained from a
standard halo-based approach. We show that our method has the same robustness
property as large-scale halo bias: baryonic physics can change the
normalization of the estimated $f_{NL}$, but cannot change whether $f_{NL}$ is
detected.
