  An impurity measures $I:{R}^k \to {R}^+$ maps a $k$-dimensional vector ${\bf
v}$ to a non-negative value $I({\bf v})$ so that the more homogeneous ${\bf
v}$, the larger its impurity. We study clustering based on impurity measures:
given a collection $V$ of $n$ many $k$-dimensional vectors and an impurity
measure $I$, the goal is to find a partition ${\cal P}$ of $V$ into $L$ groups
$V_1,\ldots,V_L$ that minimizes the total impurities of the groups in ${\cal
P}$, i.e., $I({\cal P})= \sum_{m=1}^{L} I(\sum_{{\bf v} \in V_m}{\bf v}).$
  Impurity minimization is widely used as quality assessment measure in
probability distribution clustering and in categorical clustering where it is
not possible to rely on geometric properties of the data set. However, in
contrast to the case of metric based clustering, the current knowledge of
impurity measure based clustering in terms of approximation and
inapproximability results is very limited.
  Our research contributes to fill this gap. We first present a simple linear
time algorithm that simultaneously achieves $3$-approximation for the Gini
impurity measure and $O(\log(\sum_{{\bf v} \in V} \| {\bf v}
\|_1))$-approximation for the Entropy impurity measure. Then, for the Entropy
impurity measure---where we also show that finding the optimal clustering is
strongly NP-hard---we are able to design a polynomial time
$O(\log^2(\min\{k,L\}))$-approximation algorithm. Our algorithm relies on a
nontrivial characterization of a class of clusterings that necessarily includes
a partition achieving $O(\log^2(\min\{k,L\}))$--approximation of the impurity
of the optimal partition. Remarkably, this is the first polynomial time
algorithm with approximation guarantee independent of the number of
points/vector and not relying on any restriction on the components of the
vectors for producing clusterings with minimum entropy.
