<p>It would seem to depend on the complexity of your neural network. A single perceptron is a linear separator. On the other hand, it is known that multilayer feed-forward neural networks are universal approximators (see here: <a href="http://dl.acm.org/citation.cfm?id=70408" rel="nofollow">http://dl.acm.org/citation.cfm?id=70408</a> ): i.e. they can approximate any function to arbitrary accuracy. Obviously for such a network, you should not expect to be able to express its outputs as a linear function of its inputs. </p>

CC BY-SA 3.0
2012-03-08T16:19:19.303Z
Sure, they can approximate functions to arbitrary accuracy. But the approximation that they learn is ultimately also a function, yes? So wouldn't that mean that I could write a function `f(x1, x2, ..., xn) -> (y1, y2, ..., yn) that perfectly models this approximation?
inspectorG4dget
CC BY-SA 3.0
2012-03-08T16:31:14.750Z
Of course -- a neural network represents a function. But not necessarily a linear function.
Aaron Roth
CC BY-SA 3.0
2012-03-08T16:42:46.537Z
Please correct me if I'm wrong, but do you mean "not a linear function IN 2 DIMENSIONS"? It's my understanding that it's always a linear function, possibly, though in higher dimensional spaces, which would make it super-linear in 2-D space
inspectorG4dget
CC BY-SA 3.0
2012-03-08T16:46:42.997Z
Incidentally, if you don't apply any sigmoid functions (ie. all neurons are linear) then any multilayer perceptron is also a linear function (every layer is a linear function, so you're just chaining linear functions). The point is that if your NN is a linear function, then you may as well be training a linear function (a single layer NN without sigmoid), or better yet, just calculate the optimal solution directly with least squares. What you _can_ do is compute each layer as a linear function and apply the sigmoid to the resulting vector.
Peter
CC BY-SA 3.0
2012-03-08T15:54:03.717Z
2012-03-08T15:54:03.717Z
Aaron Roth
<p>I have trained a neural network using MATLAB and am ready to deploy it into my software. Right now, I include it in my software by programming the structure of the neural network and the connection weights and doing some matrix multiplication.</p>

<p>But there has to be a better way to do this: neural networks are basically linear separators. This means that it should be possible to model the outputs as some linear function of the inputs.</p>

<p>So here's the question: given the structure of a neural network and the connection weights after training, how would I come up with the correct linear function of the inputs that models the functionality of the network?</p>

<ne.neural-evol>
Linear Function Representation of Neural Network
inspectorG4dget
# Linear Function Representation of Neural Network

(Asked by: username_0 on 2012-03-08)

I have trained a neural network using MATLAB and am ready to deploy it into my software. Right now, I include it in my software by programming the structure of the neural network and the connection weights and doing some matrix multiplication.

But there has to be a better way to do this: neural networks are basically linear separators. This means that it should be possible to model the outputs as some linear function of the inputs.

So here's the question: given the structure of a neural network and the connection weights after training, how would I come up with the correct linear function of the inputs that models the functionality of the network?

**Tags:** ne.neural-evol

---

## 1 answers

### Answer by username_1 on 2012-03-08. Score: 2 | Accepted answer

It would seem to depend on the complexity of your neural network. A single perceptron is a linear separator. On the other hand, it is known that multilayer feed-forward neural networks are universal approximators (see here: <http://dl.acm.org/citation.cfm?id=70408> ): i.e. they can approximate any function to arbitrary accuracy. Obviously for such a network, you should not expect to be able to express its outputs as a linear function of its inputs.

> **Comments:**
> **username_0**: Sure, they can approximate functions to arbitrary accuracy. But the approximation that they learn is ultimately also a function, yes? So wouldn't that mean that I could write a function `f(x1, x2, ..., xn) -> (y1, y2, ..., yn) that perfectly models this approximation?  
> **username_1**: Of course -- a neural network represents a function. But not necessarily a linear function.  
> **username_0**: Please correct me if I'm wrong, but do you mean "not a linear function IN 2 DIMENSIONS"? It's my understanding that it's always a linear function, possibly, though in higher dimensional spaces, which would make it super-linear in 2-D space  
> **username_2**: Incidentally, if you don't apply any sigmoid functions (ie. all neurons are linear) then any multilayer perceptron is also a linear function (every layer is a linear function, so you're just chaining linear functions). The point is that if your NN is a linear function, then you may as well be training a linear function (a single layer NN without sigmoid), or better yet, just calculate the optimal solution directly with least squares. What you _can_ do is compute each layer as a linear function and apply the sigmoid to the resulting vector.  

---

