<p>I think this recent paper fits your requirements. It considers biological plausibility by showing that the number of neurons required in the proposed method is within a reasonable size for the human brain, and dismisses a series of unreasonable models. Specifically, they create a neural network using the Neural Engineering Framework (NEF) and the Semantic Pointer Architecture (SPA). The network contains 2.5 million neurons to contain a network of 100,000 concepts, which would only require 14.7mm$^2$, in contrast to another model discussed in the paper, which would required 500cm$^2$.</p>

<p>It's also "high-level" enough that it does not just consider relative strengths of nodes, but applies them to analysing vocabulary semantic information. For example, one of the three tests discussed in the paper is to see if the network is "smart" enough to determine that the word "dog" is a synonym for "canine".</p>

<p>The paper itself is kind of hard to swallow if you don't have the background with the NEF and SPA to approach it, so I recommend reading either <a href="http://compneuro.uwaterloo.ca/files/publications/sharma.2016.pdf" rel="nofollow">this tutorial paper</a> to get started or "How to Build a Brain" by Chris Eliasmith.</p>

<p><strong>Reference</strong></p>

<p>Eric Crawford, Matthew Gingerich, and Chris Eliasmith. <a href="http://mindmodeling.org/cogsci2013/papers/0099/paper0099.pdf" rel="nofollow">Biologically plausible, human-scale knowledge representation</a>. In <em>35th Annual Conference of the Cognitive Science Society</em>, 412–417. 2013.</p>

CC BY-SA 3.0
2014-04-09T08:19:08.263Z
@Seanny123 your comment could be turned into a separate question on the biological feasibility of VSA. Kind of like similar but more broad questions for [biological plausibility](http://cogsci.stackexchange.com/q/644/29) and [other downsides](http://cogsci.stackexchange.com/q/599/29) and [limitations](http://cogsci.stackexchange.com/q/1300/29) of bayesian models.
Artem Kaznatcheev
CC BY-SA 3.0
2014-04-09T08:45:45.053Z
While I think (and have very limited proof) that it would be wrong to characterize SPA (which is actually being used here, which is based off of VSA) as purely probabilistic, I understand your point and will post a question once I finish the book "How to Build a Brain".
Seanny123
CC BY-SA 3.0
2014-04-09T14:15:32.620Z
@Seanny123 I wasn't suggesting characterizing it as such. I was just giving examples of questions that are "how biologically plausible is x?" or "what are the downsides of x?" and x just happened to be bayesian models because those are posts I happened to remember.
Artem Kaznatcheev
CC BY-SA 3.0
2014-02-15T16:15:57.887Z
I must say the VSA (vector symbolic architecture) and it's neural implementation doesn't strike me as being biologically plausible. I guess it is relatively more plausible than what the authors are comparing to in some sense.
Memming
CC BY-SA 3.0
2014-02-16T02:24:13.107Z
@Memming Would you care to elaborate why not and what is it specifically about VSA that you consider biologically implausible? I'm still making my way through understanding this paper and would really appreciate your perspective.
Seanny123
CC BY-SA 3.0
2014-02-14T08:07:43.013Z
2016-07-06T01:06:28.637Z
2016-07-06T01:06:28.637Z
Seanny123
<p>In my experience, the term "semantic knowledge" (or semantic memory or conceptual knowledge) is generally used to refer to knowledge of objects, word meanings, facts and people, without connection to any particular time or place. The neural basis of this kind of knowledge is more or less agreed to depend on a distributed network of cortical brain regions (e.g., Martin, 2007; Patterson, Nestor, &amp; Rogers, 2007). Tim Rogers and Jay McClelland (e.g., Rogers &amp; McClelland, 2004) are among the leaders in developing biologically plausible computational models of semantic memory (and many others are working on it as well).</p>

<p>Goals are usually put in the domain of "executive functions" or "cognitive control" and generally associated with prefrontal (and frontal) cortical regions. I know somewhat less about this area of research, but there are definitely biologically plausible models being developed, including by Botvinick and Plaut (2004) and you might find what you're looking for in the work of Munakata and O'Reilly (e.g., Munakata et al., 2011)</p>

<p>References</p>

<ul>
<li><p>Botvinick, M. M., &amp; Plaut, D. C. (2004). <a href="http://dx.doi.org/10.1037/0033-295X.111.2.395">Doing Without Schema Hierarchies: A Recurrent Connectionist Approach to Normal and Impaired Routine Sequential Action.</a> Psychological Review, 111(2), 395-429, <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.24.3773&amp;rep=rep1&amp;type=pdf">free pdf</a>.</p></li>
<li><p>Martin, A. (2007). <a href="http://dx.doi.org/10.1146/annurev.psych.57.102904.190143">The representation of object concepts in the brain.</a> Annual Review of Psychology, 58, 25-45, <a href="http://psych.stanford.edu/~jlm/pdfs/MartinAnnRevPsych07.pdf">free pdf</a>.</p></li>
<li><p>Munakata, Y., Herd, S. A., Chatham, C. H., Depue, B. E., Banich, M. T., &amp; O’Reilly, R. C. (2011). <a href="http://dx.doi.org/10.1016/j.tics.2011.07.011">A unified framework for inhibitory control.</a> Trends in Cognitive Sciences, 15(10), 453-459.</p></li>
<li><p>Patterson, K. E., Nestor, P. J., &amp; Rogers, T. T. (2007). <a href="http://dx.doi.org/10.1038/nrn2277">Where do you know what you know? The representation of semantic knowledge in the human brain.</a> Nature Reviews Neuroscience, 8, 976-987, <a href="http://brainadvantage.com/PDF/Memory%20systems%20-%20Nat%20Rev%20NeuroSci%202007%20-%201.pdf">free pdf</a>.</p></li>
<li><p>Rogers, T. T., &amp; McClelland, J. L. (2004). <a href="http://books.google.es/books?id=AmB33Uz2MVAC">Semantic Cognition: A Parallel Distributed Processing Approach.</a> Cambridge, MA: MIT Press.</p></li>
</ul>

CC BY-SA 3.0
2012-02-27T06:36:48.243Z
Great -- some new ones in there, as well as oldies I'd forgotten about.  Also re-discovering the genius of McClelland and Rumelhart's original PDP papers.  Thanks.
shanusmagnus
CC BY-SA 3.0
2012-02-24T20:37:38.607Z
2014-02-14T09:54:55.847Z
2014-02-14T09:54:55.847Z
Dan M.
<p>I'm looking for references that deal with the issue of how various kinds of semantic knowledge are (or might be) represented neurally.  Most of the discussion of this topic seems skewed by social psychologists, who talk in very high-level terms of how various semantic constructs (such as goals) might be represented based on behavioral observations (cf. Eitam &amp; Higgins, 2010).  But it's been hard to find accounts that are both closer to the metal (that consider neural plausibility) and also high-level (that treat semantics more complex than associative strength between generic nodes.)  </p>

<p>Can anyone point me to good material on this topic?</p>

<p>References:</p>

<ul>
<li>Eitam, B., &amp; Higgins, E. T. (2010). <a href="http://dx.doi.org/10.1111/j.1751-9004.2010.00309.x" rel="noreferrer">Motivation in Mental Accessibility: Relevance of a Representation (ROAR) as a New Framework.</a> Social and Personality Psychology Compass, Relevance of a Representation, 4(10), 951–967, <a href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2992320/pdf/nihms216171.pdf" rel="noreferrer">free pdf</a></li>
</ul>

<reference-request><cognitive-modeling><computational-modeling><neural-network><theoretical-neuroscience>
References for biologically plausible models of knowledge representation?
shanusmagnus
# References for biologically plausible models of knowledge representation?

(Asked by: username_0 on 2012-02-24)

I'm looking for references that deal with the issue of how various kinds of semantic knowledge are (or might be) represented neurally. Most of the discussion of this topic seems skewed by social psychologists, who talk in very high-level terms of how various semantic constructs (such as goals) might be represented based on behavioral observations (cf. Eitam & Higgins, 2010). But it's been hard to find accounts that are both closer to the metal (that consider neural plausibility) and also high-level (that treat semantics more complex than associative strength between generic nodes.)

Can anyone point me to good material on this topic?

References:

* Eitam, B., & Higgins, E. T. (2010). [Motivation in Mental Accessibility: Relevance of a Representation (ROAR) as a New Framework.](http://dx.doi.org/10.1111/j.1751-9004.2010.00309.x) Social and Personality Psychology Compass, Relevance of a Representation, 4(10), 951–967, [free pdf](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2992320/pdf/nihms216171.pdf)

**Tags:** reference-request, cognitive-modeling, computational-modeling, neural-network, theoretical-neuroscience


---

## 2 answers

### Answer by username_4 on 2012-02-24. Score: 17 | Accepted answer

In my experience, the term "semantic knowledge" (or semantic memory or conceptual knowledge) is generally used to refer to knowledge of objects, word meanings, facts and people, without connection to any particular time or place. The neural basis of this kind of knowledge is more or less agreed to depend on a distributed network of cortical brain regions (e.g., Martin, 2007; Patterson, Nestor, & Rogers, 2007). Tim Rogers and Jay McClelland (e.g., Rogers & McClelland, 2004) are among the leaders in developing biologically plausible computational models of semantic memory (and many others are working on it as well).

Goals are usually put in the domain of "executive functions" or "cognitive control" and generally associated with prefrontal (and frontal) cortical regions. I know somewhat less about this area of research, but there are definitely biologically plausible models being developed, including by Botvinick and Plaut (2004) and you might find what you're looking for in the work of Munakata and O'Reilly (e.g., Munakata et al., 2011)

References

* Botvinick, M. M., & Plaut, D. C. (2004). [Doing Without Schema Hierarchies: A Recurrent Connectionist Approach to Normal and Impaired Routine Sequential Action.](http://dx.doi.org/10.1037/0033-295X.111.2.395) Psychological Review, 111(2), 395-429, [free pdf](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.24.3773&rep=rep1&type=pdf).
* Martin, A. (2007). [The representation of object concepts in the brain.](http://dx.doi.org/10.1146/annurev.psych.57.102904.190143) Annual Review of Psychology, 58, 25-45, [free pdf](http://psych.stanford.edu/~jlm/pdfs/MartinAnnRevPsych07.pdf).
* Munakata, Y., Herd, S. A., Chatham, C. H., Depue, B. E., Banich, M. T., & O’Reilly, R. C. (2011). [A unified framework for inhibitory control.](http://dx.doi.org/10.1016/j.tics.2011.07.011) Trends in Cognitive Sciences, 15(10), 453-459.
* Patterson, K. E., Nestor, P. J., & Rogers, T. T. (2007). [Where do you know what you know? The representation of semantic knowledge in the human brain.](http://dx.doi.org/10.1038/nrn2277) Nature Reviews Neuroscience, 8, 976-987, [free pdf](http://brainadvantage.com/PDF/Memory%20systems%20-%20Nat%20Rev%20NeuroSci%202007%20-%201.pdf).
* Rogers, T. T., & McClelland, J. L. (2004). [Semantic Cognition: A Parallel Distributed Processing Approach.](http://books.google.es/books?id=AmB33Uz2MVAC) Cambridge, MA: MIT Press.

> **Comments:**
> **username_0**: Great -- some new ones in there, as well as oldies I'd forgotten about.  Also re-discovering the genius of McClelland and Rumelhart's original PDP papers.  Thanks.  

---

### Answer by username_1 on 2014-02-14. Score: 8

I think this recent paper fits your requirements. It considers biological plausibility by showing that the number of neurons required in the proposed method is within a reasonable size for the human brain, and dismisses a series of unreasonable models. Specifically, they create a neural network using the Neural Engineering Framework (NEF) and the Semantic Pointer Architecture (SPA). The network contains 2.5 million neurons to contain a network of 100,000 concepts, which would only require 14.7mm$^2$, in contrast to another model discussed in the paper, which would required 500cm$^2$.

It's also "high-level" enough that it does not just consider relative strengths of nodes, but applies them to analysing vocabulary semantic information. For example, one of the three tests discussed in the paper is to see if the network is "smart" enough to determine that the word "dog" is a synonym for "canine".

The paper itself is kind of hard to swallow if you don't have the background with the NEF and SPA to approach it, so I recommend reading either [this tutorial paper](http://compneuro.uwaterloo.ca/files/publications/sharma.2016.pdf) to get started or "How to Build a Brain" by Chris Eliasmith.

**Reference**

Eric Crawford, Matthew Gingerich, and Chris Eliasmith. [Biologically plausible, human-scale knowledge representation](http://mindmodeling.org/cogsci2013/papers/0099/paper0099.pdf). In *35th Annual Conference of the Cognitive Science Society*, 412–417. 2013.

> **Comments:**
> **username_3**: I must say the VSA (vector symbolic architecture) and it's neural implementation doesn't strike me as being biologically plausible. I guess it is relatively more plausible than what the authors are comparing to in some sense.  
> **username_1**: @username_3 Would you care to elaborate why not and what is it specifically about VSA that you consider biologically implausible? I'm still making my way through understanding this paper and would really appreciate your perspective.  
> **username_2**: @username_1 your comment could be turned into a separate question on the biological feasibility of VSA. Kind of like similar but more broad questions for [biological plausibility](http://cogsci.stackexchange.com/q/644/29) and [other downsides](http://cogsci.stackexchange.com/q/599/29) and [limitations](http://cogsci.stackexchange.com/q/1300/29) of bayesian models.  
> **username_1**: While I think (and have very limited proof) that it would be wrong to characterize SPA (which is actually being used here, which is based off of VSA) as purely probabilistic, I understand your point and will post a question once I finish the book "How to Build a Brain".  
> **username_2**: @username_1 I wasn't suggesting characterizing it as such. I was just giving examples of questions that are "how biologically plausible is x?" or "what are the downsides of x?" and x just happened to be bayesian models because those are posts I happened to remember.  

---

