  We consider the class of inverse probability weight (IPW) estimators,
including the popular Horvitz-Thompson and Hajek estimators used routinely in
survey sampling, causal inference and evidence estimation for Bayesian
computation. We focus on the 'weak paradoxes' for these estimators due to two
counterexamples by Basu [1988] and Wasserman [2004] and investigate the two
natural Bayesian answers to this problem: one based on binning and smoothing :
a 'Bayesian sieve' and the other based on a conjugate hierarchical model that
allows borrowing information via exchangeability. We compare the mean squared
errors for the two Bayesian estimators with the IPW estimators for Wasserman's
example via simulation studies on a broad range of parameter configurations. We
also prove posterior consistency for the Bayes estimators under
missing-completely-at-random assumption and show that it requires fewer
assumptions on the inclusion probabilities. We also revisit the connection
between the different problems where improved or adaptive IPW estimators will
be useful, including survey sampling, evidence estimation strategies such as
Conditional Monte Carlo, Riemannian sum, Trapezoidal rules and vertical
likelihood, as well as average treatment effect estimation in causal inference.
