<p>Computational neuroscience and neural networks are both studied on <a href="http://www.sussex.ac.uk/easy/smsc" rel="nofollow">this MSc at the University of Sussex</a>.
When I took the course in 2004/5, the Neural Networks module was compulsory, and the Computational Neuroscience was optional in the 2nd semester, so that would suggest the course designers (world leaders in biologically inspired computing) thought that studying neural networks first might aid the study of computational neuroscience.
I think some of the other subjects taught on the course would be of interest to you, e.g. evolving genetic algorithms for robot control (see Rodney Brooks – and the iRobot corporation). </p>

<p>Lastly to answer the question(!), I think you probably need some understanding of (simple) neural networks (artificial or otherwise) to understand more in-depth concepts in computational neuroscience.</p>

CC BY-SA 3.0
2014-05-22T20:43:28.767Z
2014-05-22T21:29:53.330Z
2014-05-22T21:29:53.330Z
stephendwolff
<p>Not very related insofar, unfortunately. While the original inspiration for artificial neural networks (ANN) was biological, most of the subsequent progress in ANNs for what is called "machine learning", which is usually concerned with optimizing some function, came from mathematical insights that were not based in biology; quoting <a href="https://www.frontiersin.org/articles/10.3389/fncom.2016.00094/full" rel="nofollow noreferrer">Marblestone et al.(2016)</a>:</p>

<blockquote>
  <p>The artificial neural networks now prominent in machine learning were, of course, originally inspired by neuroscience (McCulloch and Pitts, 1943). While neuroscience has continued to play a role (Cox and Dean, 2014), many of the major developments were guided by insights into the mathematics of efficient optimization, rather than neuroscientific findings (Sutskever and Martens, 2013). The field has advanced from simple linear systems (Minsky and Papert, 1972), to nonlinear networks (Haykin, 1994), to deep and recurrent networks (LeCun et al., 2015; Schmidhuber, 2015). Backpropagation of error (Werbos, 1974, 1982; Rumelhart et al., 1986) enabled neural networks to be trained efficiently, by providing an efficient means to compute the gradient with respect to the weights of a multi-layer network. Methods of training have improved to include momentum terms, better weight initializations, conjugate gradients and so forth, evolving to the current breed of networks optimized using batch-wise stochastic gradient descent. These developments have little obvious connection to neuroscience.</p>
</blockquote>

<p>Marblestone (who is an AI researcher) and colleagues do argue (in fact this is the point of their article) that </p>

<blockquote>
  <p>however, that neuroscience and machine learning are again ripe for convergence</p>
</blockquote>

<p>Their list of arguments is fairly long and I'll certainly not do them justice here, but for instance they cite a recent paper proposing to explain that <a href="https://en.wikipedia.org/wiki/Hebbian_theory" rel="nofollow noreferrer">Hebbian plasticity</a> is a form of optimization:</p>

<blockquote>
  <p>Often these types of local self-organization can also be viewed as optimizing a cost function: for example, certain forms of Hebbian plasticity can be viewed as extracting the principal components of the input, which minimizes a reconstruction error (<a href="https://arxiv.org/abs/1511.09468" rel="nofollow noreferrer">Pehlevan and Chklovskii, 2015</a>).</p>
</blockquote>

<p>Marblestone et al. dedicate a substantial amount of their paper reviewing fairly recent work that has attempt to pinpoint backpropagation (a key method for ANN's success in optimization) in the brain. The list of hypotheses how backpropagation might happen in the brain is pretty long, alas, so I won't review them here; I'm just pointing to a few papers and talks <a href="https://www.youtube.com/watch?v=VIRCybGgHts" rel="nofollow noreferrer">Hinton (2016)</a>; <a href="https://dspace.mit.edu/bitstream/handle/1721.1/100797/CBMM-Memo-036.pdf" rel="nofollow noreferrer">Liao (2015)</a> that examine how backpropagation is (or isn't) related to biological neworks or which even propose biologically inspired alternatives to be used in ANNs, e.g. <a href="https://arxiv.org/pdf/1411.6191.pdf" rel="nofollow noreferrer">Balduzzi (2014)</a>.</p>

<p>Another issue that Marblestone et al. cover in depth is how cost functions might be represented in the brain. And this meshes in with how memories and goals are represented in the brain. This is obviously a vast area of research. Since the deep biology papers are generally beyond my pay grade, I'll just point to
a recent paper (highlighted by Marblestone) proposing that a "Naïve Utility Calculus" underlies much of "Commonsense Psychology"; <a href="https://www.ncbi.nlm.nih.gov/pubmed/27388875" rel="nofollow noreferrer">Jara-Ettinger et. al (2016)</a>.</p>

CC BY-SA 4.0
2018-07-15T14:55:12.383Z
For the narrower issue with backpropagation see https://psychology.stackexchange.com/questions/16269/is-back-prop-biologically-plausible
got trolled too much this week
CC BY-SA 3.0
2018-01-07T02:58:10.063Z
2018-01-07T02:58:10.063Z
got trolled too much this week
<p>If one is more interested in understanding how algorithms in the biological brain solve problems (theoretically, particularly the mathematical aspect), and possibly in building brain-inspired computation (applied theory, particularly neurorobotics), then is it suggested to focus more on studying computational neuroscience rather than artificial neural nets/machine learning? It seems the latter one is more oriented toward any algorithms just to solve problems via computer simulations without the constraints of biological brains, though there are still large areas for theory of machine learning and artificial neural nets.</p>

<theoretical-neuroscience><computational-modeling><neural-network>
How distantly related are research in computational neuroscience and neural networks/machine learning?
Xingdong Zuo
# How distantly related are research in computational neuroscience and neural networks/machine learning?

(Asked by: username_0 on 2014-05-21)

If one is more interested in understanding how algorithms in the biological brain solve problems (theoretically, particularly the mathematical aspect), and possibly in building brain-inspired computation (applied theory, particularly neurorobotics), then is it suggested to focus more on studying computational neuroscience rather than artificial neural nets/machine learning? It seems the latter one is more oriented toward any algorithms just to solve problems via computer simulations without the constraints of biological brains, though there are still large areas for theory of machine learning and artificial neural nets.

**Tags:** theoretical-neuroscience, computational-modeling, neural-network


---

## 2 answers

### Answer by username_1 on 2014-05-22. Score: 4

Computational neuroscience and neural networks are both studied on [this MSc at the University of Sussex](http://www.sussex.ac.uk/easy/smsc).
When I took the course in 2004/5, the Neural Networks module was compulsory, and the Computational Neuroscience was optional in the 2nd semester, so that would suggest the course designers (world leaders in biologically inspired computing) thought that studying neural networks first might aid the study of computational neuroscience.
I think some of the other subjects taught on the course would be of interest to you, e.g. evolving genetic algorithms for robot control (see Rodney Brooks – and the iRobot corporation).

Lastly to answer the question(!), I think you probably need some understanding of (simple) neural networks (artificial or otherwise) to understand more in-depth concepts in computational neuroscience.


---

### Answer by username_2 on 2018-01-07. Score: 1

Not very related insofar, unfortunately. While the original inspiration for artificial neural networks (ANN) was biological, most of the subsequent progress in ANNs for what is called "machine learning", which is usually concerned with optimizing some function, came from mathematical insights that were not based in biology; quoting [Marblestone et al.(2016)](https://www.frontiersin.org/articles/10.3389/fncom.2016.00094/full):

> The artificial neural networks now prominent in machine learning were, of course, originally inspired by neuroscience (McCulloch and Pitts, 1943). While neuroscience has continued to play a role (Cox and Dean, 2014), many of the major developments were guided by insights into the mathematics of efficient optimization, rather than neuroscientific findings (Sutskever and Martens, 2013). The field has advanced from simple linear systems (Minsky and Papert, 1972), to nonlinear networks (Haykin, 1994), to deep and recurrent networks (LeCun et al., 2015; Schmidhuber, 2015). Backpropagation of error (Werbos, 1974, 1982; Rumelhart et al., 1986) enabled neural networks to be trained efficiently, by providing an efficient means to compute the gradient with respect to the weights of a multi-layer network. Methods of training have improved to include momentum terms, better weight initializations, conjugate gradients and so forth, evolving to the current breed of networks optimized using batch-wise stochastic gradient descent. These developments have little obvious connection to neuroscience.

Marblestone (who is an AI researcher) and colleagues do argue (in fact this is the point of their article) that

> however, that neuroscience and machine learning are again ripe for convergence

Their list of arguments is fairly long and I'll certainly not do them justice here, but for instance they cite a recent paper proposing to explain that [Hebbian plasticity](https://en.wikipedia.org/wiki/Hebbian_theory) is a form of optimization:

> Often these types of local self-organization can also be viewed as optimizing a cost function: for example, certain forms of Hebbian plasticity can be viewed as extracting the principal components of the input, which minimizes a reconstruction error ([Pehlevan and Chklovskii, 2015](https://arxiv.org/abs/1511.09468)).

Marblestone et al. dedicate a substantial amount of their paper reviewing fairly recent work that has attempt to pinpoint backpropagation (a key method for ANN's success in optimization) in the brain. The list of hypotheses how backpropagation might happen in the brain is pretty long, alas, so I won't review them here; I'm just pointing to a few papers and talks [Hinton (2016)](https://www.youtube.com/watch?v=VIRCybGgHts); [Liao (2015)](https://dspace.mit.edu/bitstream/handle/1721.1/100797/CBMM-Memo-036.pdf) that examine how backpropagation is (or isn't) related to biological neworks or which even propose biologically inspired alternatives to be used in ANNs, e.g. [Balduzzi (2014)](https://arxiv.org/pdf/1411.6191.pdf).

Another issue that Marblestone et al. cover in depth is how cost functions might be represented in the brain. And this meshes in with how memories and goals are represented in the brain. This is obviously a vast area of research. Since the deep biology papers are generally beyond my pay grade, I'll just point to
a recent paper (highlighted by Marblestone) proposing that a "Naïve Utility Calculus" underlies much of "Commonsense Psychology"; [Jara-Ettinger et. al (2016)](https://www.ncbi.nlm.nih.gov/pubmed/27388875).

> **Comments:**
> **username_2**: For the narrower issue with backpropagation see https://psychology.stackexchange.com/questions/16269/is-back-prop-biologically-plausible  

---

