  We introduce a balloon estimator in a generalized expectation-maximization
method for estimating all parameters of a Gaussian mixture model given one data
sample per mixture component. Instead of limiting explicitly the model size,
this regularization strategy yields low-complexity sparse models where the
number of effective mixture components reduces with an increase of a smoothing
probability parameter $\mathbf{P>0}$. This semi-parametric method bridges from
non-parametric adaptive kernel density estimation (KDE) to parametric ordinary
least-squares when $\mathbf{P=1}$. Experiments show that simpler sparse mixture
models retain the level of details present in the adaptive KDE solution.
