<p>I would be very surprised if a backpropagation-trained multilayer FFN spontaneously learned concepts of shape. It's not that the notion of "shape" isn't representable by a FFN (given enough layers, <em>anything</em> is). It may even be the case that the globally optimal configuration <em>does</em> capture something about shape. But there must be a strong local minimum about the exact-image replicator, meaning that typical backpropagation (i.e., gradient descent) algorithms will get stuch in such local minima. Unless you've specifically primed your FFN to learn shape, I would be very surprised if it did.</p>

CC BY-SA 3.0
2011-06-26T14:31:58.093Z
So it is possible to teach it shape by feeding different images of the same shape? (I know, in my original setup I only fed it 1 image per shape)
Cray
CC BY-SA 3.0
2011-06-27T06:26:22.470Z
In the connectionist (as opposed to symbolic or rule-based) framework, it's hard to say whether or not the system has "learned shape". The only test is tautological: see if it manages to recognize novel shapes of previously seen letters. Your idea of training on different images of the same letter sounds like it should work (I believe that's how they trained the zipcode reader FFN).
Aryeh
CC BY-SA 3.0
2011-06-26T08:49:50.767Z
2011-06-26T08:49:50.767Z
Aryeh
<p>(Feel free to suggest a better title) (same with tags, none of relevant tags exist yet, and I only have 101)</p>

<p>Let's say I have a perceptron with an optimal number of layers and optimal number of neurons, which I have taught to recognize an alphabet which was printed in a certain font, with the same settings and resolution. Now if I feed it a symbol from that set, but say, printed in another (but similar) font, or with the same font, but rotated 5 degrees, or shifted 2 pixels to the left, or scaled a 5 percent, and so on, will this system recognize this modified symbol, or at least have an easier time learning to recognize this symbol? (than a system without any prior learning)</p>

<p>Basically, what is the point of back-propagating neural networks? Is it
1) a way to recognize <em>shapes</em>, and effectively filter noise in images of those shapes, in form of different rotations, scales and other disortions
or
2) just a relatively simple and fast way to recognize a limited number of bitmaps of those shapes, and not being able to recognize the shape in another bitmap if that particular bitmap was never taught to the network.</p>

CC BY-SA 3.0
2011-06-26T00:47:56.560Z
The question is probably more suitable for MetaOptimize Q&A, you can find a link in the FAQ.
Kaveh
<machine-learning>
Does a multilayer perceptron taught by simple backprop learn the shape of a character or the exact image of a character?
Cray
# Does a multilayer perceptron taught by simple backprop learn the shape of a character or the exact image of a character?

(Asked by: username_0 on 2011-06-25)

(Feel free to suggest a better title) (same with tags, none of relevant tags exist yet, and I only have 101)

Let's say I have a perceptron with an optimal number of layers and optimal number of neurons, which I have taught to recognize an alphabet which was printed in a certain font, with the same settings and resolution. Now if I feed it a symbol from that set, but say, printed in another (but similar) font, or with the same font, but rotated 5 degrees, or shifted 2 pixels to the left, or scaled a 5 percent, and so on, will this system recognize this modified symbol, or at least have an easier time learning to recognize this symbol? (than a system without any prior learning)

Basically, what is the point of back-propagating neural networks? Is it
1) a way to recognize *shapes*, and effectively filter noise in images of those shapes, in form of different rotations, scales and other disortions
or
2) just a relatively simple and fast way to recognize a limited number of bitmaps of those shapes, and not being able to recognize the shape in another bitmap if that particular bitmap was never taught to the network.

**Tags:** machine-learning
> **Comments:**
> **username_1**: The question is probably more suitable for MetaOptimize Q&A, you can find a link in the FAQ.  

---

## 1 answers

### Answer by username_2 on 2011-06-26. Score: 1

I would be very surprised if a backpropagation-trained multilayer FFN spontaneously learned concepts of shape. It's not that the notion of "shape" isn't representable by a FFN (given enough layers, *anything* is). It may even be the case that the globally optimal configuration *does* capture something about shape. But there must be a strong local minimum about the exact-image replicator, meaning that typical backpropagation (i.e., gradient descent) algorithms will get stuch in such local minima. Unless you've specifically primed your FFN to learn shape, I would be very surprised if it did.

> **Comments:**
> **username_0**: So it is possible to teach it shape by feeding different images of the same shape? (I know, in my original setup I only fed it 1 image per shape)  
> **username_2**: In the connectionist (as opposed to symbolic or rule-based) framework, it's hard to say whether or not the system has "learned shape". The only test is tautological: see if it manages to recognize novel shapes of previously seen letters. Your idea of training on different images of the same letter sounds like it should work (I believe that's how they trained the zipcode reader FFN).  

---

