<p>Convolution is a technical term, and comes from the signal processing field. Unfortunately, it is easily confused with "convoluting". Effectively, "to convolve" means to pick a small "window", slide it across your signal, and compute the same function for each step.</p>

<p>In CNNs, when applied to classifying, for example images, it means taking a small window, of say 3x3 pixels, sliding it across the whole image, and feeding the small sections of the larger image under the window into a neural network. The outputs of the network are then combined into a new layer and can then be fed into another NN layer for classification.</p>

<p><a href="https://i.sstatic.net/HMDnR.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/HMDnR.jpg" alt="Convolutional Neural Network"></a></p>

<p>In the example above, the window is 3x3 pixels wide, and the step size is 1. Depending on the application, it's possible to have larger/smaller windows and step sizes.</p>

<p>The resulting layer can be said to be the <em>result of convolving the original image with a neural network</em>.</p>

<p>When using convolution filters in, say Photoshop, it's using a similar method where it takes sliding windows of pixels and computes a function (for example to achieve blur, the average RGB values of the window pixels) and creates a new image out of those. You can then say that <em>convolving an image with a pixel average function results in a blurred image</em>.</p>

CC BY-SA 3.0
2016-05-19T21:29:42.903Z
And there are different functions that can be swapped in for the pixel-average function, resulting in things like edge detection. (I've heard this function called "the filter kernel".
DJG
CC BY-SA 3.0
2016-05-19T21:54:52.817Z
Minx, yes exactly
Justas
CC BY-SA 4.0
2022-07-19T03:36:19.080Z
@Justas In the diagram, there is a circle below the circle with the $\sum$ in it. This circle has a stretched-out "S" symbol. What does it mean?
user110391
CC BY-SA 4.0
2022-07-19T20:06:23.110Z
@user110391 the S symbol signifies a non-linear activation function through which the sum is passed. Depending on the use case, the S could be the sigmoid, tanh, or RELU function, among other possibilities.
Justas
CC BY-SA 4.0
2022-07-19T21:02:02.557Z
@Justas Thanks :)
user110391
CC BY-SA 3.0
2016-05-18T20:11:58.797Z
2016-05-18T20:11:58.797Z
Justas
<p>The <a href="https://en.wikipedia.org/wiki/Types_of_artificial_neural_networks" rel="nofollow">"Types of Neural Networks" wiki page</a> doesn't even have the word "convolution" in it, and yet there's an <a href="http://cs.stanford.edu/people/karpathy/convnetjs/index.html" rel="nofollow">entire Javascript library</a> based on Convolutional Neural Networks (CNN). </p>

<p>So, what makes CNNs "convolutional"? When I hear "convolution", I think of either smearing/blurring together (as in the "convolve" tool in programs like Photoshop), or complexity/tangling (e.g., "a long and unnecessarily convoluted explanation").</p>

<p>When I think of neural networks, I think of networks with distinct layers, where neurons from one layer only connect to neurons of the next layer. To me, "convolutional" suggests that the neurons are more intricately connected somehow—or "tangled up". But I see nothing in the javascript library's documentation to suggest that this is the case.</p>

<p>The background section in the <a href="http://arxiv.org/abs/1602.09046" rel="nofollow">"On Complex Valued Convolutional Neural
Networks"</a> master thesis suggests that CNNs are what has made neural networks actually useful for tasks like computer vision and face recognition. Why remains unclear, however, the term "convolution operation" was mentioned.</p>

<p><sub>
Guberman, N. (2016). <a href="http://arxiv.org/abs/1602.09046" rel="nofollow">On Complex Valued Convolutional Neural Networks.</a> arXiv preprint arXiv:1602.09046.
</sub></p>

CC BY-SA 3.0
2016-03-14T20:03:02.893Z
It is the [mathematical meaning](https://en.wikipedia.org/wiki/Convolution) which is related to the Photoshop convolve tool.
StrongBad
CC BY-SA 3.0
2016-03-15T16:40:53.457Z
i.e. each layer of the network performs a particular convolution on its inputs.
honi
<neural-network>
What does "Convolutional" signify in "Convolutional Neural Network"?
DJG
# What does "Convolutional" signify in "Convolutional Neural Network"?

(Asked by: username_0 on 2016-03-14)

The ["Types of Neural Networks" wiki page](https://en.wikipedia.org/wiki/Types_of_artificial_neural_networks) doesn't even have the word "convolution" in it, and yet there's an [entire Javascript library](http://cs.stanford.edu/people/karpathy/convnetjs/index.html) based on Convolutional Neural Networks (CNN).

So, what makes CNNs "convolutional"? When I hear "convolution", I think of either smearing/blurring together (as in the "convolve" tool in programs like Photoshop), or complexity/tangling (e.g., "a long and unnecessarily convoluted explanation").

When I think of neural networks, I think of networks with distinct layers, where neurons from one layer only connect to neurons of the next layer. To me, "convolutional" suggests that the neurons are more intricately connected somehow—or "tangled up". But I see nothing in the javascript library's documentation to suggest that this is the case.

The background section in the ["On Complex Valued Convolutional Neural
Networks"](http://arxiv.org/abs/1602.09046) master thesis suggests that CNNs are what has made neural networks actually useful for tasks like computer vision and face recognition. Why remains unclear, however, the term "convolution operation" was mentioned.

Guberman, N. (2016). [On Complex Valued Convolutional Neural Networks.](http://arxiv.org/abs/1602.09046) arXiv preprint arXiv:1602.09046.

**Tags:** neural-network

> **Comments:**
> **username_1**: It is the [mathematical meaning](https://en.wikipedia.org/wiki/Convolution) which is related to the Photoshop convolve tool.  
> **username_2**: i.e. each layer of the network performs a particular convolution on its inputs.  

---

## 1 answers

### Answer by username_3 on 2016-05-18. Score: 4 | Accepted answer

Convolution is a technical term, and comes from the signal processing field. Unfortunately, it is easily confused with "convoluting". Effectively, "to convolve" means to pick a small "window", slide it across your signal, and compute the same function for each step.

In CNNs, when applied to classifying, for example images, it means taking a small window, of say 3x3 pixels, sliding it across the whole image, and feeding the small sections of the larger image under the window into a neural network. The outputs of the network are then combined into a new layer and can then be fed into another NN layer for classification.

[![Convolutional Neural Network](https://i.sstatic.net/HMDnR.jpg)](https://i.sstatic.net/HMDnR.jpg)

In the example above, the window is 3x3 pixels wide, and the step size is 1. Depending on the application, it's possible to have larger/smaller windows and step sizes.

The resulting layer can be said to be the *result of convolving the original image with a neural network*.

When using convolution filters in, say Photoshop, it's using a similar method where it takes sliding windows of pixels and computes a function (for example to achieve blur, the average RGB values of the window pixels) and creates a new image out of those. You can then say that *convolving an image with a pixel average function results in a blurred image*.

> **Comments:**
> **username_0**: And there are different functions that can be swapped in for the pixel-average function, resulting in things like edge detection. (I've heard this function called "the filter kernel".  
> **username_3**: Minx, yes exactly  
> **username_4**: @username_3 In the diagram, there is a circle below the circle with the $\sum$ in it. This circle has a stretched-out "S" symbol. What does it mean?  
> **username_3**: @username_4 the S symbol signifies a non-linear activation function through which the sum is passed. Depending on the use case, the S could be the sigmoid, tanh, or RELU function, among other possibilities.  
> **username_4**: @username_3 Thanks :)  

---

