<p>Recently Tom Griffiths and I worked on the problem of how to use distributional cues from the raw sensory data given by the environment to form representational units (features). The basic idea is to form a probabilistic model over possible feature representations. The model uses the observed data and a bias towards simpler feature representations (those with fewer features) to infer a feature representation. The tricky thing is forming a probabilistic model  over the complex, infinite-dimensional space of feature representations and once that is defined, the practical part of inferring the feature representation given a set of observed objects. </p>

<p>For more details on how to do this, what it means theoretically, and experiments testing some predictions of the approach, take a look at our recent article in cognitive psychology, <a href="http://dx.doi.org/10.1016/j.cogpsych.2011.08.002" rel="nofollow">doi</a>.</p>

<p>Also, if you're interested, we've done some work looking at inferring transformation-invariant features, <a href="http://cocosci.berkeley.edu/joe/tibp.pdf" rel="nofollow">pdf</a>. We're nearly ready to submit a larger article on this line of work. Send me an email if you're interested.</p>

CC BY-SA 3.0
2012-03-12T03:47:28.630Z
2012-03-12T03:47:28.630Z
Joe Austerweil
<p>I am interested in the creation of chunks (aka configural nodes) from smaller chunks and input features (only interested in System 1 cognition).</p>

<p>Unitization studies (e.g. Goldstone (<a href="http://cogprints.org/909/3/doodlesold.pdf" rel="nofollow">pdf</a>)), suggest that we start with generic features, and slowly combine them into more specific chunks. As we do this unitization, we wouldn't store all of the learned combinations --- a mere 10 input features can be combined in 3.6 million configurations!</p>

<p>Simon suggested generic elements get overwritten by more specific ones at each exposure. but what would you do with the predictions/rules learned about a given chunk? Transfer them to the newly created specific chunk? What if they do not apply at that more specific level?</p>

<p>Nosofsky was suggesting a rule-plus-exception model, where the more generic elements are always stored and their predictions are recorded as general rules; When those predictions are broken, more specific chunks are stored to explain the exceptions to these rules. What does it mean for predictions to be broken? In a probabilistic environment like ours, sometimes a prediction is correct, sometimes it isn't -- updating the weight of the prediction seems much more reasonable than deeming it invalid in favor of an 'exception'.</p>

<p>When do I chunk two co-occurring features? When do I chunk those two with a third? Do I delete the 2-feature chunk in favor of the 3-feature? What do I do with the memories associated with the deleted chunk?</p>

CC BY-SA 3.0
2012-03-07T19:27:50.967Z
Welcome to the site Vlad! This is a great question, but it'd be even better if you could link to some of your sources. Also, you might want to change the name of your question to something more descriptive such as a combination of some of your closing remarks (e.g. How does chunk formation avoid a combinatorial explosion?).
zergylord
CC BY-SA 3.0
2012-03-12T12:21:40.900Z
Also, please try to summarize the main question at the end of the post in bold. Something like "How do we avoid the combinatorial explosion of chunking?" if that is your main question.
Artem Kaznatcheev
<learning><cognitive-modeling>
Combinatorial woes
vdv
# Combinatorial woes

(Asked by: username_0 on 2012-03-07)

I am interested in the creation of chunks (aka configural nodes) from smaller chunks and input features (only interested in System 1 cognition).

Unitization studies (e.g. Goldstone ([pdf](http://cogprints.org/909/3/doodlesold.pdf))), suggest that we start with generic features, and slowly combine them into more specific chunks. As we do this unitization, we wouldn't store all of the learned combinations --- a mere 10 input features can be combined in 3.6 million configurations!

Simon suggested generic elements get overwritten by more specific ones at each exposure. but what would you do with the predictions/rules learned about a given chunk? Transfer them to the newly created specific chunk? What if they do not apply at that more specific level?

Nosofsky was suggesting a rule-plus-exception model, where the more generic elements are always stored and their predictions are recorded as general rules; When those predictions are broken, more specific chunks are stored to explain the exceptions to these rules. What does it mean for predictions to be broken? In a probabilistic environment like ours, sometimes a prediction is correct, sometimes it isn't -- updating the weight of the prediction seems much more reasonable than deeming it invalid in favor of an 'exception'.

When do I chunk two co-occurring features? When do I chunk those two with a third? Do I delete the 2-feature chunk in favor of the 3-feature? What do I do with the memories associated with the deleted chunk?

**Tags:** learning, cognitive-modeling

> **Comments:**
> **username_1**: Welcome to the site Vlad! This is a great question, but it'd be even better if you could link to some of your sources. Also, you might want to change the name of your question to something more descriptive such as a combination of some of your closing remarks (e.g. How does chunk formation avoid a combinatorial explosion?).  
> **username_2**: Also, please try to summarize the main question at the end of the post in bold. Something like "How do we avoid the combinatorial explosion of chunking?" if that is your main question.  

---

## 1 answers

### Answer by username_3 on 2012-03-12. Score: 3

Recently Tom Griffiths and I worked on the problem of how to use distributional cues from the raw sensory data given by the environment to form representational units (features). The basic idea is to form a probabilistic model over possible feature representations. The model uses the observed data and a bias towards simpler feature representations (those with fewer features) to infer a feature representation. The tricky thing is forming a probabilistic model over the complex, infinite-dimensional space of feature representations and once that is defined, the practical part of inferring the feature representation given a set of observed objects.

For more details on how to do this, what it means theoretically, and experiments testing some predictions of the approach, take a look at our recent article in cognitive psychology, [doi](http://dx.doi.org/10.1016/j.cogpsych.2011.08.002).

Also, if you're interested, we've done some work looking at inferring transformation-invariant features, [pdf](http://cocosci.berkeley.edu/joe/tibp.pdf). We're nearly ready to submit a larger article on this line of work. Send me an email if you're interested.


---

