<p>I have an $n \times m$ matrix of data $\mathbf{D}$ as well as a $k$-partition $P$ of $n$ indices each representing a row in a dataset. Assuming an arbitrary clustering algorithm $A$, I would like to find a subset $F$ of $\{1,\ldots, m\}$, representing indices of the columns of $D$, such that applying $A$ to only the columns indexed by $F$, an $n \times |F|$ matrix, minimizes the average variation of information (VI) between $P$ and the output of $A$. Another way of saying this is that I would like to be able to find the subspace on which $\mathbf{D}$ is most informative about $P$. </p>

<p>(That the metric is VI or that the statistic taken over multiple runs is an average is not necessarily important. I'm looking for a general solution, but if a solution that specifies either the metric or the clustering algorithm exists, I'd be interested.)</p>

<p>Attempting to build the distribution of (normalized) variation of information values by brute force is not practical since the number of possible subsets of $\{1,\ldots, m\}$ is the Stirling number (of the second kind) $S(m, 2)$, which obviously gets monstrous as $m$ gets to any interesting size.</p>

CC BY-SA 3.0
2012-09-13T17:37:27.590Z
The general problem your specific problem falls under is called "subspace clustering" - have you looked into that literature ? Since you're agnostic on some of the choices of cost function, you might find something appropriate there.
Suresh Venkat
CC BY-SA 3.0
2012-09-13T22:32:10.193Z
I have not looked there. Thanks a lot, Suresh. I suspected there was a (possibly large) literature on this sort of problem. I will check it out.
aaronsteven
<machine-learning><clustering><data-mining>
Finding most informative feature subsets given dataset, clustering algorithm and gold standard partition
aaronsteven
# Finding most informative feature subsets given dataset, clustering algorithm and gold standard partition

(Asked by: username_0 on 2012-09-13)

I have an $n \times m$ matrix of data $\mathbf{D}$ as well as a $k$-partition $P$ of $n$ indices each representing a row in a dataset. Assuming an arbitrary clustering algorithm $A$, I would like to find a subset $F$ of $\{1,\ldots, m\}$, representing indices of the columns of $D$, such that applying $A$ to only the columns indexed by $F$, an $n \times |F|$ matrix, minimizes the average variation of information (VI) between $P$ and the output of $A$. Another way of saying this is that I would like to be able to find the subspace on which $\mathbf{D}$ is most informative about $P$.

(That the metric is VI or that the statistic taken over multiple runs is an average is not necessarily important. I'm looking for a general solution, but if a solution that specifies either the metric or the clustering algorithm exists, I'd be interested.)

Attempting to build the distribution of (normalized) variation of information values by brute force is not practical since the number of possible subsets of $\{1,\ldots, m\}$ is the Stirling number (of the second kind) $S(m, 2)$, which obviously gets monstrous as $m$ gets to any interesting size.

**Tags:** machine-learning, clustering, data-mining

> **Comments:**
> **username_1**: The general problem your specific problem falls under is called "subspace clustering" - have you looked into that literature ? Since you're agnostic on some of the choices of cost function, you might find something appropriate there.  
> **username_0**: I have not looked there. Thanks a lot, Suresh. I suspected there was a (possibly large) literature on this sort of problem. I will check it out.  

---

## 0 answers

