<p>The weight matrix is typically considered to be a strength of connectivity metric between nodes in the parallel computronium model that neural networks are based on. That fact is fairly evident when you investigate ANN learning algorithms.</p>

<p>For instance, the backpropagation algorithm for feedforward networks is designed to strengthen a string of associations between layers, thus forming a state vector to represent the optimal solution.</p>

<p>The association seems even more evident in Hopfield networks. A training algorithm in a Hopfield net is designed to adjust the weight matrix until the network state converges to a stability point. Certain stabilities represent certain associative memories; similar to the brain.</p>

<p>However, I think forming too strong of a connection between the current neural network models and the functions of the biological brain might actually be detrimental to your understanding of the former. Neural networks are not designed to work like the brain, they're designed to be a parallel architecture. It just so happens that the brain is also a parallel architecture, but learning in the brain is so different from any ANN model that it makes very little sense to compare the two on such a fundamental level.</p>

CC BY-SA 3.0
2015-08-11T21:21:36.200Z
Very informative. Thank you. I understand that most people are not interested in detailed similarities and differences between real and artificial neural nets. I am because I believe this understanding gives clues as to how the state of the art will be advanced. I should have been more clear about that in my question.
Alex Ryan
CC BY-SA 3.0
2015-08-11T00:09:50.563Z
2015-08-11T00:09:50.563Z
jay
<p>Historically, neuroscientists have not thought of myelination to be the biological analogue of weights in an artificial neural network (but see below!). </p>

<p>Instead, <strong>the strength of synaptic connections</strong> seems to be the most likely analogue. Here, strength refers mostly to the size and of the voltage response in the receiving neuron due to the release of neurotransmitter in the sending neuron. So, for example, Neuron A is connected to Neuron B, and then A releases the neurotransmitter glutamate and causes a 1 millivolt response in B. But then another neuron, Neuron C may be connected to Neuron D and when it releases glutamate it might only cause a 0.5 millivolt response--not as strong as the A->B connection. <em>Importantly, the strengths of synaptic connections change due to experience, and this is thought to (in part) underlie learning, similarly to how changes in weights in an artificial neural network are the mechanism of artificial learning.</em></p>

<p>All this said, your point about myelination is interesting in light of recent work: an <a href="http://www.sciencemag.org/content/346/6207/318" rel="nofollow">October 2014 paper</a>, published in <em>Science</em> showed that production of myelin (in the form of oligodendrocytes) was necessary for motor learning in mice. </p>

CC BY-SA 3.0
2015-08-11T17:43:08.147Z
2015-08-11T20:52:16.227Z
2015-08-11T20:52:16.227Z
Chelonian
<p>The weights in an <a href="https://en.wikipedia.org/wiki/Artificial_neural_network" rel="noreferrer">artificial neural network</a> are an <em>approximation</em> of multiple processes combined that take place in biological neurons. <a href="https://en.wikipedia.org/wiki/Myelin" rel="noreferrer">Myelination</a> plays a role, but not a major one. Weights in artificial neural networks can be positive or negative numbers. </p>

<p><strong>Weight magnitude</strong>. The magnitude of a weight is analogous to a combination of increased <a href="https://en.wikipedia.org/wiki/Dendrite" rel="noreferrer">dendritic</a> connections between neurons, number of <a href="https://en.wikipedia.org/wiki/Synapse" rel="noreferrer">synapses</a> between their dendrites, density of <a href="https://en.wikipedia.org/wiki/Neurotransmitter" rel="noreferrer">neurotransmitter</a> <a href="https://en.wikipedia.org/wiki/Neurotransmitter_receptor" rel="noreferrer">receptors</a> on the <a href="https://en.wikipedia.org/wiki/Chemical_synapse" rel="noreferrer">post-synaptic terminals</a>, as well as increased <a href="https://en.wikipedia.org/wiki/Synaptic_vesicle" rel="noreferrer">neurotransmitter vesicle</a> formation and fusion on the pre-synaptic terminals.</p>

<p><strong>Positive weights</strong>. Positive weights are analogous to the pre-synaptic terminals of the synapses releasing excitatory neurotransmitters (i.e. <a href="https://en.wikipedia.org/wiki/Glutamic_acid" rel="noreferrer">glutamate</a>). They make it <em>more</em> likely the receiving cell will fire an <a href="https://en.wikipedia.org/wiki/Action_potential" rel="noreferrer">action potential</a>.</p>

<p><strong>Negative weights</strong>. Negative weights are analogous to inhibitory neurotransmitters (i.e. <a href="https://en.wikipedia.org/wiki/Gamma-Aminobutyric_acid" rel="noreferrer">GABA</a>) being released at the synapse. They make it <em>less</em> likely the receiving cell will fire an action potential.</p>

<p>Myelination</p>

<p>Myelination increases the distance that action potentials can travel down the axon. The <a href="https://en.wikipedia.org/wiki/Membrane_potential" rel="noreferrer">membrane voltage potential</a> decays much closer to the cell body if the axon is not myelinated. One analogy is that of a garden hose. If the axon is not myelinated, then the garden hose is leaky with holes, and less water pressure (that caries water pressure waves, the action potentials) makes it to the end of the hose. </p>

<p>However, increased myelination is not necessarily analogous to an increased weight. For example, if there are few dendrites between two neurons, with few synaptic connections, then myelination would have little effect.</p>

<p>Learning</p>

<p>You also mentioned learning. During learning, the <a href="https://en.wikipedia.org/wiki/Backpropagation" rel="noreferrer">weights in an artificial neuron increase or decrease</a>. In biological neurons, learning takes place at multiple scales and areas (i.e. <a href="https://en.wikipedia.org/wiki/Nonsynaptic_plasticity" rel="noreferrer">non-synaptic</a> and <a href="https://en.wikipedia.org/wiki/Synaptic_plasticity" rel="noreferrer">synaptic</a>). At first, during increasing connection strength, you may see increased vesicle fusion, then increased neurotransmitter receptors, then new synaptic boutons, and new <a href="https://en.wikipedia.org/wiki/Dendritic_spine" rel="noreferrer">dendrites</a>. When "unlearning", or forgetting, these processes happen in reverse.</p>

<p>Overall, a weight in artificial neurons clump a lot of biological complexity into one number that only crudely approximates the degree of connection strength between two biological neurons.</p>

CC BY-SA 4.0
2019-01-09T14:10:13.673Z
Hi @Justas , what's your opinion on weights initialization on ANNs? check my question https://ai.stackexchange.com/questions/9908/how-do-biological-neurons-weights-get-initialized/
Leo Gallucci
CC BY-SA 3.0
2015-08-11T17:54:40.440Z
2015-08-11T17:54:40.440Z
Justas
<p>In artificial neural networks the connections between neurons are a assigned numbers called "weights" or "parameters". As new data is fed into the neural net, these weights change. This is how the neural net "learns".</p>

<p>I'm trying to understand what these weights might correspond to in a real  brain. The idea occurred to me that it might represent the strength of the myelination on the connections between real neurons.</p>

<p>Is this a correct understanding?</p>

<p>Does the degree of myelineation along an axon vary over time as the neural net processes more experiences?</p>

<computational-modeling><neural-network>
What do the weights of an artificial neural network represent in biological neurons?
Alex Ryan
# What do the weights of an artificial neural network represent in biological neurons?

(Asked by: username_0 on 2015-08-10)

In artificial neural networks the connections between neurons are a assigned numbers called "weights" or "parameters". As new data is fed into the neural net, these weights change. This is how the neural net "learns".

I'm trying to understand what these weights might correspond to in a real brain. The idea occurred to me that it might represent the strength of the myelination on the connections between real neurons.

Is this a correct understanding?

Does the degree of myelineation along an axon vary over time as the neural net processes more experiences?

**Tags:** computational-modeling, neural-network


---

## 3 answers

### Answer by username_3 on 2015-08-11. Score: 7 | Accepted answer

The weights in an [artificial neural network](https://en.wikipedia.org/wiki/Artificial_neural_network) are an *approximation* of multiple processes combined that take place in biological neurons. [Myelination](https://en.wikipedia.org/wiki/Myelin) plays a role, but not a major one. Weights in artificial neural networks can be positive or negative numbers.

**Weight magnitude**. The magnitude of a weight is analogous to a combination of increased [dendritic](https://en.wikipedia.org/wiki/Dendrite) connections between neurons, number of [synapses](https://en.wikipedia.org/wiki/Synapse) between their dendrites, density of [neurotransmitter](https://en.wikipedia.org/wiki/Neurotransmitter) [receptors](https://en.wikipedia.org/wiki/Neurotransmitter_receptor) on the [post-synaptic terminals](https://en.wikipedia.org/wiki/Chemical_synapse), as well as increased [neurotransmitter vesicle](https://en.wikipedia.org/wiki/Synaptic_vesicle) formation and fusion on the pre-synaptic terminals.

**Positive weights**. Positive weights are analogous to the pre-synaptic terminals of the synapses releasing excitatory neurotransmitters (i.e. [glutamate](https://en.wikipedia.org/wiki/Glutamic_acid)). They make it *more* likely the receiving cell will fire an [action potential](https://en.wikipedia.org/wiki/Action_potential).

**Negative weights**. Negative weights are analogous to inhibitory neurotransmitters (i.e. [GABA](https://en.wikipedia.org/wiki/Gamma-Aminobutyric_acid)) being released at the synapse. They make it *less* likely the receiving cell will fire an action potential.

Myelination

Myelination increases the distance that action potentials can travel down the axon. The [membrane voltage potential](https://en.wikipedia.org/wiki/Membrane_potential) decays much closer to the cell body if the axon is not myelinated. One analogy is that of a garden hose. If the axon is not myelinated, then the garden hose is leaky with holes, and less water pressure (that caries water pressure waves, the action potentials) makes it to the end of the hose.

However, increased myelination is not necessarily analogous to an increased weight. For example, if there are few dendrites between two neurons, with few synaptic connections, then myelination would have little effect.

Learning

You also mentioned learning. During learning, the [weights in an artificial neuron increase or decrease](https://en.wikipedia.org/wiki/Backpropagation). In biological neurons, learning takes place at multiple scales and areas (i.e. [non-synaptic](https://en.wikipedia.org/wiki/Nonsynaptic_plasticity) and [synaptic](https://en.wikipedia.org/wiki/Synaptic_plasticity)). At first, during increasing connection strength, you may see increased vesicle fusion, then increased neurotransmitter receptors, then new synaptic boutons, and new [dendrites](https://en.wikipedia.org/wiki/Dendritic_spine). When "unlearning", or forgetting, these processes happen in reverse.

Overall, a weight in artificial neurons clump a lot of biological complexity into one number that only crudely approximates the degree of connection strength between two biological neurons.

> **Comments:**
> **username_4**: Hi @username_3 , what's your opinion on weights initialization on ANNs? check my question https://ai.stackexchange.com/questions/9908/how-do-biological-neurons-weights-get-initialized/  

---

### Answer by username_1 on 2015-08-11. Score: 4

The weight matrix is typically considered to be a strength of connectivity metric between nodes in the parallel computronium model that neural networks are based on. That fact is fairly evident when you investigate ANN learning algorithms.

For instance, the backpropagation algorithm for feedforward networks is designed to strengthen a string of associations between layers, thus forming a state vector to represent the optimal solution.

The association seems even more evident in Hopfield networks. A training algorithm in a Hopfield net is designed to adjust the weight matrix until the network state converges to a stability point. Certain stabilities represent certain associative memories; similar to the brain.

However, I think forming too strong of a connection between the current neural network models and the functions of the biological brain might actually be detrimental to your understanding of the former. Neural networks are not designed to work like the brain, they're designed to be a parallel architecture. It just so happens that the brain is also a parallel architecture, but learning in the brain is so different from any ANN model that it makes very little sense to compare the two on such a fundamental level.

> **Comments:**
> **username_0**: Very informative. Thank you. I understand that most people are not interested in detailed similarities and differences between real and artificial neural nets. I am because I believe this understanding gives clues as to how the state of the art will be advanced. I should have been more clear about that in my question.  

---

### Answer by username_2 on 2015-08-11. Score: 0

Historically, neuroscientists have not thought of myelination to be the biological analogue of weights in an artificial neural network (but see below!).

Instead, **the strength of synaptic connections** seems to be the most likely analogue. Here, strength refers mostly to the size and of the voltage response in the receiving neuron due to the release of neurotransmitter in the sending neuron. So, for example, Neuron A is connected to Neuron B, and then A releases the neurotransmitter glutamate and causes a 1 millivolt response in B. But then another neuron, Neuron C may be connected to Neuron D and when it releases glutamate it might only cause a 0.5 millivolt response--not as strong as the A->B connection. *Importantly, the strengths of synaptic connections change due to experience, and this is thought to (in part) underlie learning, similarly to how changes in weights in an artificial neural network are the mechanism of artificial learning.*

All this said, your point about myelination is interesting in light of recent work: an [October 2014 paper](http://www.sciencemag.org/content/346/6207/318), published in *Science* showed that production of myelin (in the form of oligodendrocytes) was necessary for motor learning in mice.


---

