<p>This is not really a question related much to biological neural networks, rather it's a concept from <a href="https://en.wikipedia.org/wiki/Artificial_neural_network#Organization" rel="nofollow noreferrer">artificial neural networks that are arranged in layered structure</a>.</p>
<p>In a layered artificial neural network, &quot;neurons&quot; of one layer receive input <em>only</em> from the previous layer, and give output <em>only</em> to the subsequent layer. This creates a computationally efficient feed-forward structure: there's no complicated recurrence, once you know the activities of one layer you can calculate the activities of the next layer by a single multiplication step between the weight matrix and activity vector of the previous layer, followed by a sum for each unit.</p>
<p>From the README of the software you link to:</p>
<blockquote>
<p>There are 3 main levels of structure: Network, Layer and Prjn (projection). The network calls methods on its Layers, and Layers iterate over both Neuron data structures (which have only a minimal set of methods) and the Prjns, to implement the relevant computations. The Prjn fully manages everything about a projection of connectivity between two layers, including the full list of Syanpse elements in the connection. ... The Layer also has a set of Pool elements, one for each level at which inhibition is computed</p>
</blockquote>
<p>So, while the neurons in this network may have some biological aspects like integration over time, it seems they are arranged into layers like a classical ANN, information is shared between layers through a &quot;Prjn&quot; object that manages the &quot;synapses&quot; between two layers. Additionally, each layer seems to have its own inhibitory unit. I assume the way this is modeled is that every neuron in a layer gives input to an inhibitory unit, which then reduces the activity within that layer as sort of a gain control. Again, while they may be biologically motivated, these are all artificial constructions for a computational model chosen by the authors.</p>
<p>While you could conceivably still map these connections all into an NxN matrix, everything would be zero except for upper triangular matrices corresponding to each pair of layers. Nothing about hierarchical organization says you can't put it in a larger matrix, just that it doesn't necessarily make sense to do so. For example, you could put students in a classroom into a longer list by school, that doesn't mean there isn't an underlying organization of students within classroom just because you abstracted out that level.</p>
<p>While there are also layered structures in biological networks (like <a href="https://en.wikipedia.org/wiki/Neocortex" rel="nofollow noreferrer">neocortex</a>) these layers do not function like artificial neural network layers, as neurons within layers are strongly connected to each other, and while there is some evidence for a &quot;feed-forward&quot; pathway (roughly, layer 4 to 2/3 to 5 to 6), this is only a moderate bias in connectivity and only a minority of connections actually follow this canonical pattern.</p>

CC BY-SA 4.0
2022-09-07T15:29:37.447Z
i am asking about biological networks! the book Computational Cognitive Neuroscience  and the  leabra software are both doing biological simulations, **not AI/ML stuff**
em1971
CC BY-SA 4.0
2022-09-07T15:35:14.167Z
@DingRuiqi From what I see from the software, it's at best biologically inspired. They're choosing to organize things in layers, like typical ANNs. They're describing their computational architecture when they say it's organized in layers. They may not be using the strict feedforward network I described, but it's still an organizational choice they're making.
Bryan Krause
CC BY-SA 4.0
2022-09-07T15:36:21.507Z
For example copied from the README: "There are 3 main levels of structure: Network, Layer and Prjn (projection). The network calls methods on its Layers, and Layers iterate over both Neuron data structures (which have only a minimal set of methods) and the Prjns, to implement the relevant computations. The Prjn fully manages everything about a projection of connectivity between two layers, including the full list of Syanpse elements in the connection. ... The Layer also has a set of Pool elements, one for each level at which inhibition is computed "
Bryan Krause
CC BY-SA 4.0
2022-09-07T14:38:42.020Z
2022-09-07T16:53:38.660Z
2022-09-07T16:53:38.660Z
Bryan Krause
<p>in this book <a href="https://github.com/CompCogNeuro/ed4" rel="nofollow noreferrer">Computational Cognitive Neuroscience</a> and its corresponding software <a href="https://github.com/emer/leabra" rel="nofollow noreferrer">leabra</a></p>
<blockquote>
<p>The Leabra Algorithm:
Leabra stands for Local, Error-driven and Associative, <strong>Biologically Realistic</strong> Algorithm, and it implements a balance between error-driven (backpropagation) and associative (Hebbian) learning on top of a <strong>biologically-based</strong> point-neuron activation function with inhibitory competition dynamics (either via inhibitory interneurons or an approximation thereof), which produce k-Winners-Take-All (kWTA) sparse distributed representations.</p>
</blockquote>
<p>a <strong>biological</strong> neural network is composed of 3 hierarchies: neuron -&gt; layer -&gt; network</p>
<p>Why is it not: neuron -&gt; network ?</p>
<p>suppose you have a brain with <span class="math-container">$10$</span> neurons, then the neural network can be described as a <span class="math-container">$10\times 10$</span> connectivity matrix, and that's all, right?</p>
<p>Why do you need, for example, to divide the <span class="math-container">$10$</span> neurons into <span class="math-container">$2$</span> layers, and each layer has <span class="math-container">$5$</span> neurons, and how do you even distinguish them? (how do you know which neuron belongs to which layer?)</p>

CC BY-SA 4.0
2022-09-07T16:41:48.323Z
I see you've changed the title. Nonetheless, you link to an algorithmic implementation of a biologically-inspired network. They are describing their software. "Biologically-based" I think is throwing you off here, they are still talking about an artificial network. They have chosen to divide their network into layers, because they chose to do it, so that's what their software does.
Bryan Krause
CC BY-SA 4.0
2022-09-07T16:42:49.693Z
You can read about biological layers in the Wikipedia page for Neocortex that I linked in my answer, but you will not find that these biological (anatomical) layers have very much to do with the layers in the model you link to.
Bryan Krause
CC BY-SA 4.0
2022-11-03T02:40:19.893Z
It's usually called full Boltzman machine without layers in your case which is known to be of exponential computational complexity, usually one needs to add some layers to restrict complexity to be practical. Of course you need much experience and knowledge to optimally decide number of layers and number of neurons in each layer for a specific problem, usually more layers are preferred for difficult multi-dimensional input.
cinch
<neural-network>
what is a layer in a **biological** neural network?
em1971
# what is a layer in a **biological** neural network?

(Asked by: username_0 on 2022-09-07)

in this book [Computational Cognitive Neuroscience](https://github.com/CompCogNeuro/ed4) and its corresponding software [leabra](https://github.com/emer/leabra)

> The Leabra Algorithm:
> Leabra stands for Local, Error-driven and Associative, **Biologically Realistic** Algorithm, and it implements a balance between error-driven (backpropagation) and associative (Hebbian) learning on top of a **biologically-based** point-neuron activation function with inhibitory competition dynamics (either via inhibitory interneurons or an approximation thereof), which produce k-Winners-Take-All (kWTA) sparse distributed representations.

a **biological** neural network is composed of 3 hierarchies: neuron -> layer -> network

Why is it not: neuron -> network ?

suppose you have a brain with $10$ neurons, then the neural network can be described as a $10\times 10$ connectivity matrix, and that's all, right?

Why do you need, for example, to divide the $10$ neurons into $2$ layers, and each layer has $5$ neurons, and how do you even distinguish them? (how do you know which neuron belongs to which layer?)

**Tags:** neural-network

> **Comments:**
> **username_1**: I see you've changed the title. Nonetheless, you link to an algorithmic implementation of a biologically-inspired network. They are describing their software. "Biologically-based" I think is throwing you off here, they are still talking about an artificial network. They have chosen to divide their network into layers, because they chose to do it, so that's what their software does.  
> **username_1**: You can read about biological layers in the Wikipedia page for Neocortex that I linked in my answer, but you will not find that these biological (anatomical) layers have very much to do with the layers in the model you link to.  
> **username_2**: It's usually called full Boltzman machine without layers in your case which is known to be of exponential computational complexity, usually one needs to add some layers to restrict complexity to be practical. Of course you need much experience and knowledge to optimally decide number of layers and number of neurons in each layer for a specific problem, usually more layers are preferred for difficult multi-dimensional input.  

---

## 1 answers

### Answer by username_1 on 2022-09-07. Score: 2

This is not really a question related much to biological neural networks, rather it's a concept from [artificial neural networks that are arranged in layered structure](https://en.wikipedia.org/wiki/Artificial_neural_network#Organization).

In a layered artificial neural network, "neurons" of one layer receive input *only* from the previous layer, and give output *only* to the subsequent layer. This creates a computationally efficient feed-forward structure: there's no complicated recurrence, once you know the activities of one layer you can calculate the activities of the next layer by a single multiplication step between the weight matrix and activity vector of the previous layer, followed by a sum for each unit.

From the README of the software you link to:

> There are 3 main levels of structure: Network, Layer and Prjn (projection). The network calls methods on its Layers, and Layers iterate over both Neuron data structures (which have only a minimal set of methods) and the Prjns, to implement the relevant computations. The Prjn fully manages everything about a projection of connectivity between two layers, including the full list of Syanpse elements in the connection. ... The Layer also has a set of Pool elements, one for each level at which inhibition is computed

So, while the neurons in this network may have some biological aspects like integration over time, it seems they are arranged into layers like a classical ANN, information is shared between layers through a "Prjn" object that manages the "synapses" between two layers. Additionally, each layer seems to have its own inhibitory unit. I assume the way this is modeled is that every neuron in a layer gives input to an inhibitory unit, which then reduces the activity within that layer as sort of a gain control. Again, while they may be biologically motivated, these are all artificial constructions for a computational model chosen by the authors.

While you could conceivably still map these connections all into an NxN matrix, everything would be zero except for upper triangular matrices corresponding to each pair of layers. Nothing about hierarchical organization says you can't put it in a larger matrix, just that it doesn't necessarily make sense to do so. For example, you could put students in a classroom into a longer list by school, that doesn't mean there isn't an underlying organization of students within classroom just because you abstracted out that level.

While there are also layered structures in biological networks (like [neocortex](https://en.wikipedia.org/wiki/Neocortex)) these layers do not function like artificial neural network layers, as neurons within layers are strongly connected to each other, and while there is some evidence for a "feed-forward" pathway (roughly, layer 4 to 2/3 to 5 to 6), this is only a moderate bias in connectivity and only a minority of connections actually follow this canonical pattern.

> **Comments:**
> **username_0**: i am asking about biological networks! the book Computational Cognitive Neuroscience  and the  leabra software are both doing biological simulations, **not AI/ML stuff**  
> **username_1**: @DingRuiqi From what I see from the software, it's at best biologically inspired. They're choosing to organize things in layers, like typical ANNs. They're describing their computational architecture when they say it's organized in layers. They may not be using the strict feedforward network I described, but it's still an organizational choice they're making.  
> **username_1**: For example copied from the README: "There are 3 main levels of structure: Network, Layer and Prjn (projection). The network calls methods on its Layers, and Layers iterate over both Neuron data structures (which have only a minimal set of methods) and the Prjns, to implement the relevant computations. The Prjn fully manages everything about a projection of connectivity between two layers, including the full list of Syanpse elements in the connection. ... The Layer also has a set of Pool elements, one for each level at which inhibition is computed "  

---

