<p>Artificial neural networks (ANNs) are mathematical constructs, originally designed to approximate biological neurons. Each "neuron" is a relatively simple element --- for example, summing its inputs and applying a threshold to the result, to determine the output of that "neuron".</p>

<p>Several decades of research went into discovering how to build network architectures using these mathematical constructs, and how to automatically set the weighting on each of the connections between the neurons to perform a wide range of tasks. For example, ANNs can do things like recognition of hand-written digits.</p>

<p>A "biological neural network" would refer to any group of connected biological nerve cells. Your brain is a biological neural network, so is a number of neurons grown together in a dish so that they form synaptic connections. The term "biological neural network" is not very precise; it doesn't define a particular biological structure.</p>

<p>In the same way, an ANN can mean any of a large number of mathematical "neuron"-like constructs, connected in any of a large number of ways, to perform any of a large number of tasks.</p>

CC BY-SA 3.0
2017-02-01T19:02:54.053Z
ANN's are simplified Mathematical Models of the actual Bilogical Networks---which simulate the performance of actual biological network.
b.sahu
CC BY-SA 3.0
2014-07-25T12:52:42.980Z
2014-07-25T12:52:42.980Z
Dylan Richard Muir
<p>I'm studying computer science at KIT (Karlsruhe Institute of Technology, Germany) specializing in Machine Learning and a minor in Mathematics. I am not a biologist.</p>
<p>An artificial neural network is basically a mathematical function. It is built from simple functions which have parameters (numbers) which get adjusted (learned). One example of such a function is:</p>
<p><span class="math-container">$$\varphi(x_1, \dots, x_n) = \frac{1}{1+e^{\sum_{i=1}^n w_i x_i}} \text{ with } w_i \in \mathbb{R}, x_i \in \mathbb{R}$$</span></p>
<p>As you can see, there are parameter <span class="math-container">$w_i$</span> which can be set to any number. The <span class="math-container">$x_i$</span> are the inputs. But those inputs themselves can be the output of another function <span class="math-container">$\varphi$</span>.</p>
<p>A common way to visualize those simple functions is</p>
<p><a href="https://i.sstatic.net/eBPgU.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/eBPgU.png" alt="enter image description here" /></a></p>
<p>There are thousands of those simple functions (also called &quot;neurons&quot;) which build an artificial neural network. This results in a single function which has <strong>millions of parameters</strong> in the biggest networks. Neural networks use relatively simple topologies which can often be expressed in a tree structure:</p>
<p><a href="https://i.sstatic.net/mgIPe.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/mgIPe.png" alt="enter image description here" /></a></p>
<p>Every single circle is one neuron (also called &quot;perceptron&quot; in some cases).</p>
<p>You have lots of training data, <strong>train once a long time, then use it often</strong>. A long time often means several days.</p>
<p>In an artificial neural network (ANN), the signal is send <strong>synchronously</strong>.</p>
<hr />
<p>As far as I know, the human brain isn't fully understood by now. From what I've heard, the basic building blocks are also called neurons. They look like that:</p>
<p><a href="https://i.sstatic.net/FkbqB.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/FkbqB.jpg" alt="enter image description here" /></a></p>
<p>The idea is that axon terminals of one neuron, which are closer to the axon hillock of another neuron have more influence on the output of the signal. But the relationship is different. It is a process which is much better described by differential equations, not something simple as the equation for artificial neurons above. The neurons send their signals <strong>asynchronously</strong>. According to <a href="https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons" rel="nofollow noreferrer">Wikipedia</a>, humans have 86 000 000 000 neurons. They are certainly restricted in how they can be connected, but there are also certainly <strong>many billions of parameters</strong> which can be adjusted in &quot;standard human brains&quot;.</p>
<p>By now, we don't know how humans really (exactly, in all details) learn. This is certainly a key difference. But we don't learn once for a long time and stop learning then. We are <strong>continuously learning very short times</strong>.</p>
<p>See also: <a href="http://arxiv.org/abs/1511.09030" rel="nofollow noreferrer">On-line Recognition of Handwritten Mathematical Symbols</a>, Chapter 4.3 for more details.</p>
<h2>TL;DR and some more facts</h2>
<ul>
<li><strong>Signal transport and processing</strong>: The human brain works asynchronously, ANNs work synchronously.</li>
<li><strong>Parameter count</strong>: Humans have many billions (<a href="http://www.businessinsider.de/a-google-ai-godfather-says-machines-could-match-human-abilities-in-5-years-2016-3?r=US&amp;IR=T" rel="nofollow noreferrer">businessinsider.de</a> writes &quot;1,000 trillion&quot;) of adjustable parameters / 86Bn neurons (<a href="https://www.theguardian.com/science/blog/2012/feb/28/how-many-neurons-human-brain" rel="nofollow noreferrer">source</a>). Even the most complicated ANNs only have several million learnable parameters.</li>
<li><strong>Training algorithm</strong>: ANNs use Gradient Descent for learning. Human brains use something different (but we don't know what)</li>
<li><strong>Processing speed</strong>: Single biological neurons are slow, while standard neurons in ANNs are fast. (Yes, I did not confuse the two. The artificial ones are faster. And I ignore spiking networks as I didn't see a single example where they are actually useful)</li>
<li><strong>Topology</strong>: Biological neural networks have complicated topologies, while ANNs are often in a tree structure (I am aware of recurrent networks - they are still pretty simple from their structure)</li>
<li><strong>Power consumption</strong>: Biological neural networks use very little power compared to artificial networks</li>
<li>Biological networks usually don't stop / start learning. ANNs have different fitting (train) and prediction (evaluate) phases.</li>
<li><strong>Field of application</strong>: ANNs are specialized. They can only perform one task. They might be perfect at playing chess, but they fail at playing go (or vice versa). Biological neural networks can learn completely new tasks.</li>
<li>ANNs need a target function. You need to tell them what is good and what is bad. I'm not sure if you can compare this to e.g. pain in biological networks.</li>
</ul>

CC BY-SA 4.0
2020-12-20T09:36:17.373Z
You say "Even the most complicated ANNs only have several million learnable parameters." I doubt this. I think Google does use neural networks or similar structures of much more parameters (probably more than in a human brain, at expense of using giant amounts of electricity to power giant computers).
porton
CC BY-SA 4.0
2020-12-20T11:09:06.030Z
@porton Let's look at the [ai.googleblog.com](https://ai.googleblog.com/). The most recent article (3 days old) refers to the [Transformer-XL article](https://arxiv.org/pdf/1901.02860.pdf). Table 4 mentioned that Google uses a 800M network. The biggest one mentioned uses 33B parameters. They got closer in the past 4 years, but are still orders of magnitudes away from trillions of parameters.
Martin Thoma
CC BY-SA 4.0
2020-12-20T11:12:14.570Z
@porton having said that ... do you have any evidence to support your believes? Did you even try to substanciate / research it?
Martin Thoma
CC BY-SA 4.0
2020-12-20T12:23:11.950Z
It's not a belief, it's an opinion. I though of the PageRank algorithm (that was the Google's ranking algorithm in the first "version" of Google, I think something like this is still used as a small part of the big Google algorithm). It looks like (I am not sure) for me that PageRank behaves somehow similar to a neural network. Newer PR variants may be even more similar to neural networks that the original PR. Google says it has 30 trillion pages in the index, this is the number of "neurons" (if they have the "right" to be called neurons) in this net.
porton
CC BY-SA 4.0
2020-12-20T12:46:01.097Z
@porton Try to estimate the necessary amount of compute / RAM / data to make good use of such a big network.
Martin Thoma
CC BY-SA 4.0
2020-12-20T13:27:54.737Z
Google's energy consumption has increased over the last few years, reaching 10.6 terawatt hours in 2018.
porton
CC BY-SA 4.0
2016-03-22T14:20:49.443Z
2020-12-20T11:10:37.647Z
2020-12-20T11:10:37.647Z
Martin Thoma
<p>I am an Statistics student at University of Warwick (incoming Stanford University) and I have an interest in explaining machine learning concepts in a <strong>non-mathematical/non-technical</strong> way.</p>

<h1>Biological Neural Network</h1>

<p>Our brain has a large network of interlinked neurons, which act as a highway for information to be transmitted from point A to point B. To send different kinds of information from A to B, the brain activates a different sets of neurons, and so essentially uses a different route to get from A to B. This is how a typical neuron might look like.
<img src="https://annalyzin.files.wordpress.com/2016/03/neuron-illustrated.png?w=1050" alt="neuron"></p>

<p>At each neuron, its dendrites receive incoming signals sent by other neurons. If the neuron receives a high enough level of signals within a certain period of time, the neuron sends an electrical pulse into the terminals. These outgoing signals are then received by other neurons.</p>

<h1>Artificial Neural Network</h1>

<p>The ANN model is modelled after the biological neural network (and hence its namesake). Similarly, in the ANN model, we have an input node (in this example we give it a handwritten image of the number 6), and an output node, which is the digit that the program recognized. </p>

<p><img src="https://annalyzin.files.wordpress.com/2016/03/ann-1.png?w=1044" alt="Artificial Neural Network overview"></p>

<p><em>A simple Artificial Neural Network map, showing two scenarios with two different inputs but with the same output. Activated neurons along the path are shown in red.</em></p>

<p>The main characteristics of an ANN is as such:</p>

<p><strong>Step 1.</strong> When the input node is given an image, it activates a unique set of neurons in the first layer, starting a chain reaction that would pave a unique path to the output node. In Scenario 1, neurons A, B, and D are activated in layer 1.</p>

<p><strong>Step 2.</strong> The activated neurons send signals to every connected neuron in the next layer. This directly affects which neurons are activated in the next layer. In Scenario 1, neuron A sends a signal to E and G, neuron B sends a signal to E, and neuron D sends a signal to F and G.</p>

<p><strong>Step 3.</strong> In the next layer, each neuron is governed by a rule on what combinations of received signals would activate the neuron (rules are trained when we give the ANN program training data, i.e. images of handwritten digits and the correct answer). In Scenario 1, neuron E is activated by the signals from A and B. However, for neuron F and G, their neurons’ rules tell them that they have not received the right signals to be activated, and hence they remains grey.</p>

<p><strong>Step 4.</strong> Steps 2-3 are repeated for all the remaining layers (it is possible for the model to have more than 2 layers), until we are left with the output node.</p>

<p><strong>Step 5.</strong> The output node deduces the correct digit based on signals received from neurons in the layer directly preceding it (layer 2). Each combination of activated neurons in layer 2 leads to one solution, though each solution can be represented by different combinations of activated neurons. In Scenarios 1 &amp; 2, two images given to the input. Because the images are different, the network activates a different set of neurons to get from the input to the output. However, the output is still able to recognise that both images are “6”.</p>

<p>Read the full post here, which includes some examples of handwritten digits that were used: <a href="https://annalyzin.wordpress.com/2016/03/13/how-do-computers-recognise-handwriting-using-artificial-neural-networks/" rel="noreferrer">https://annalyzin.wordpress.com/2016/03/13/how-do-computers-recognise-handwriting-using-artificial-neural-networks/</a></p>

CC BY-SA 3.0
2016-03-26T18:02:23.637Z
2016-03-26T18:02:23.637Z
Kenneth Soo
<p>I read that neural networks are of two types:<br>
a) Biological neural networks<br>
b) Artificial neural networks (or ANN)</p>

<p>I read, "Neural Networks are models of biological neural structures," and the biological neural structure referred to the brain in the context of that text. Is the brain called a biological neural network? If not, what is the difference between the two terms?</p>

CC BY-SA 3.0
2014-07-23T21:47:35.703Z
The term is used differently in different fields. The brain is a (biological) neural network: it's a network of neurons. Artificial neural networks, usually just referred to as neural networks, are computer simulations which process information in a way similar to how we think the brain does it. I'll add some links here in the morning.
Eoin
<terminology><neural-network>
What is the difference between biological and artificial neural networks?
meowthecat
# What is the difference between biological and artificial neural networks?

(Asked by: username_0 on 2014-07-23)

I read that neural networks are of two types:  
a) Biological neural networks  
b) Artificial neural networks (or ANN)

I read, "Neural Networks are models of biological neural structures," and the biological neural structure referred to the brain in the context of that text. Is the brain called a biological neural network? If not, what is the difference between the two terms?

**Tags:** terminology, neural-network

> **Comments:**
> **username_1**: The term is used differently in different fields. The brain is a (biological) neural network: it's a network of neurons. Artificial neural networks, usually just referred to as neural networks, are computer simulations which process information in a way similar to how we think the brain does it. I'll add some links here in the morning.  

---

## 3 answers

### Answer by username_6 on 2016-03-26. Score: 6 | Accepted answer

I am an Statistics student at University of Warwick (incoming Stanford University) and I have an interest in explaining machine learning concepts in a **non-mathematical/non-technical** way.

# Biological Neural Network

Our brain has a large network of interlinked neurons, which act as a highway for information to be transmitted from point A to point B. To send different kinds of information from A to B, the brain activates a different sets of neurons, and so essentially uses a different route to get from A to B. This is how a typical neuron might look like.
![neuron](https://annalyzin.files.wordpress.com/2016/03/neuron-illustrated.png?w=1050)

At each neuron, its dendrites receive incoming signals sent by other neurons. If the neuron receives a high enough level of signals within a certain period of time, the neuron sends an electrical pulse into the terminals. These outgoing signals are then received by other neurons.

# Artificial Neural Network

The ANN model is modelled after the biological neural network (and hence its namesake). Similarly, in the ANN model, we have an input node (in this example we give it a handwritten image of the number 6), and an output node, which is the digit that the program recognized.

![Artificial Neural Network overview](https://annalyzin.files.wordpress.com/2016/03/ann-1.png?w=1044)

*A simple Artificial Neural Network map, showing two scenarios with two different inputs but with the same output. Activated neurons along the path are shown in red.*

The main characteristics of an ANN is as such:

**Step 1.** When the input node is given an image, it activates a unique set of neurons in the first layer, starting a chain reaction that would pave a unique path to the output node. In Scenario 1, neurons A, B, and D are activated in layer 1.

**Step 2.** The activated neurons send signals to every connected neuron in the next layer. This directly affects which neurons are activated in the next layer. In Scenario 1, neuron A sends a signal to E and G, neuron B sends a signal to E, and neuron D sends a signal to F and G.

**Step 3.** In the next layer, each neuron is governed by a rule on what combinations of received signals would activate the neuron (rules are trained when we give the ANN program training data, i.e. images of handwritten digits and the correct answer). In Scenario 1, neuron E is activated by the signals from A and B. However, for neuron F and G, their neurons’ rules tell them that they have not received the right signals to be activated, and hence they remains grey.

**Step 4.** Steps 2-3 are repeated for all the remaining layers (it is possible for the model to have more than 2 layers), until we are left with the output node.

**Step 5.** The output node deduces the correct digit based on signals received from neurons in the layer directly preceding it (layer 2). Each combination of activated neurons in layer 2 leads to one solution, though each solution can be represented by different combinations of activated neurons. In Scenarios 1 & 2, two images given to the input. Because the images are different, the network activates a different set of neurons to get from the input to the output. However, the output is still able to recognise that both images are “6”.

Read the full post here, which includes some examples of handwritten digits that were used: <https://annalyzin.wordpress.com/2016/03/13/how-do-computers-recognise-handwriting-using-artificial-neural-networks/>


---

### Answer by username_2 on 2014-07-25. Score: 15

Artificial neural networks (ANNs) are mathematical constructs, originally designed to approximate biological neurons. Each "neuron" is a relatively simple element --- for example, summing its inputs and applying a threshold to the result, to determine the output of that "neuron".

Several decades of research went into discovering how to build network architectures using these mathematical constructs, and how to automatically set the weighting on each of the connections between the neurons to perform a wide range of tasks. For example, ANNs can do things like recognition of hand-written digits.

A "biological neural network" would refer to any group of connected biological nerve cells. Your brain is a biological neural network, so is a number of neurons grown together in a dish so that they form synaptic connections. The term "biological neural network" is not very precise; it doesn't define a particular biological structure.

In the same way, an ANN can mean any of a large number of mathematical "neuron"-like constructs, connected in any of a large number of ways, to perform any of a large number of tasks.

> **Comments:**
> **username_3**: ANN's are simplified Mathematical Models of the actual Bilogical Networks---which simulate the performance of actual biological network.  

---

### Answer by username_4 on 2016-03-22. Score: 13

I'm studying computer science at KIT (Karlsruhe Institute of Technology, Germany) specializing in Machine Learning and a minor in Mathematics. I am not a biologist.

An artificial neural network is basically a mathematical function. It is built from simple functions which have parameters (numbers) which get adjusted (learned). One example of such a function is:

$$\varphi(x\_1, \dots, x\_n) = \frac{1}{1+e^{\sum\_{i=1}^n w\_i x\_i}} \text{ with } w\_i \in \mathbb{R}, x\_i \in \mathbb{R}$$

As you can see, there are parameter $w\_i$ which can be set to any number. The $x\_i$ are the inputs. But those inputs themselves can be the output of another function $\varphi$.

A common way to visualize those simple functions is

[![enter image description here](https://i.sstatic.net/eBPgU.png)](https://i.sstatic.net/eBPgU.png)

There are thousands of those simple functions (also called "neurons") which build an artificial neural network. This results in a single function which has **millions of parameters** in the biggest networks. Neural networks use relatively simple topologies which can often be expressed in a tree structure:

[![enter image description here](https://i.sstatic.net/mgIPe.png)](https://i.sstatic.net/mgIPe.png)

Every single circle is one neuron (also called "perceptron" in some cases).

You have lots of training data, **train once a long time, then use it often**. A long time often means several days.

In an artificial neural network (ANN), the signal is send **synchronously**.

---

As far as I know, the human brain isn't fully understood by now. From what I've heard, the basic building blocks are also called neurons. They look like that:

[![enter image description here](https://i.sstatic.net/FkbqB.jpg)](https://i.sstatic.net/FkbqB.jpg)

The idea is that axon terminals of one neuron, which are closer to the axon hillock of another neuron have more influence on the output of the signal. But the relationship is different. It is a process which is much better described by differential equations, not something simple as the equation for artificial neurons above. The neurons send their signals **asynchronously**. According to [Wikipedia](https://en.wikipedia.org/wiki/List_of_animals_by_number_of_neurons), humans have 86 000 000 000 neurons. They are certainly restricted in how they can be connected, but there are also certainly **many billions of parameters** which can be adjusted in "standard human brains".

By now, we don't know how humans really (exactly, in all details) learn. This is certainly a key difference. But we don't learn once for a long time and stop learning then. We are **continuously learning very short times**.

See also: [On-line Recognition of Handwritten Mathematical Symbols](http://arxiv.org/abs/1511.09030), Chapter 4.3 for more details.

## TL;DR and some more facts

* **Signal transport and processing**: The human brain works asynchronously, ANNs work synchronously.
* **Parameter count**: Humans have many billions ([businessinsider.de](http://www.businessinsider.de/a-google-ai-godfather-says-machines-could-match-human-abilities-in-5-years-2016-3?r=US&IR=T) writes "1,000 trillion") of adjustable parameters / 86Bn neurons ([source](https://www.theguardian.com/science/blog/2012/feb/28/how-many-neurons-human-brain)). Even the most complicated ANNs only have several million learnable parameters.
* **Training algorithm**: ANNs use Gradient Descent for learning. Human brains use something different (but we don't know what)
* **Processing speed**: Single biological neurons are slow, while standard neurons in ANNs are fast. (Yes, I did not confuse the two. The artificial ones are faster. And I ignore spiking networks as I didn't see a single example where they are actually useful)
* **Topology**: Biological neural networks have complicated topologies, while ANNs are often in a tree structure (I am aware of recurrent networks - they are still pretty simple from their structure)
* **Power consumption**: Biological neural networks use very little power compared to artificial networks
* Biological networks usually don't stop / start learning. ANNs have different fitting (train) and prediction (evaluate) phases.
* **Field of application**: ANNs are specialized. They can only perform one task. They might be perfect at playing chess, but they fail at playing go (or vice versa). Biological neural networks can learn completely new tasks.
* ANNs need a target function. You need to tell them what is good and what is bad. I'm not sure if you can compare this to e.g. pain in biological networks.

> **Comments:**
> **username_5**: You say "Even the most complicated ANNs only have several million learnable parameters." I doubt this. I think Google does use neural networks or similar structures of much more parameters (probably more than in a human brain, at expense of using giant amounts of electricity to power giant computers).  
> **username_4**: @username_5 Let's look at the [ai.googleblog.com](https://ai.googleblog.com/). The most recent article (3 days old) refers to the [Transformer-XL article](https://arxiv.org/pdf/1901.02860.pdf). Table 4 mentioned that Google uses a 800M network. The biggest one mentioned uses 33B parameters. They got closer in the past 4 years, but are still orders of magnitudes away from trillions of parameters.  
> **username_4**: @username_5 having said that ... do you have any evidence to support your believes? Did you even try to substanciate / research it?  
> **username_5**: It's not a belief, it's an opinion. I though of the PageRank algorithm (that was the Google's ranking algorithm in the first "version" of Google, I think something like this is still used as a small part of the big Google algorithm). It looks like (I am not sure) for me that PageRank behaves somehow similar to a neural network. Newer PR variants may be even more similar to neural networks that the original PR. Google says it has 30 trillion pages in the index, this is the number of "neurons" (if they have the "right" to be called neurons) in this net.  
> **username_4**: @username_5 Try to estimate the necessary amount of compute / RAM / data to make good use of such a big network.  
> **username_5**: Google's energy consumption has increased over the last few years, reaching 10.6 terawatt hours in 2018.  

---

