  We employ a parameter-free distribution estimation framework where estimators
are random distributions and utilize the Kullback-Leibler (KL) divergence as a
loss function. Wu and Vos [J. Statist. Plann. Inference 142 (2012) 1525-1536]
show that when an estimator obtained from an i.i.d. sample is viewed as a
random distribution, the KL risk of the estimator decomposes in a fashion
parallel to the mean squared error decomposition when the estimator is a
real-valued random variable. In this paper, we explore how conditional versions
of distribution expectation ($E^{\dagger}$) can be defined so that a
distribution version of the Rao-Blackwell theorem holds. We define
distributional expectation and variance ($V^{\dagger}$) that also provide a
decomposition of KL risk in exponential and mixture families. For exponential
families, we show that the maximum likelihood estimator (viewed as a random
distribution) is distribution unbiased and is the unique uniformly minimum
distribution variance unbiased (UMV$^{\dagger}$U) estimator. Furthermore, we
show that the MLE is robust against model specification in that if the true
distribution does not belong to the exponential family, the MLE is
UMV$^{\dagger}$U for the KL projection of the true distribution onto the
exponential families provided these two distribution have the same expectation
for the canonical statistic. To allow for estimators taking values outside of
the exponential family, we include results for KL projection and define an
extended projection to accommodate the non-existence of the MLE for families
having discrete sample space. Illustrative examples are provided.
