<p>No, you don't need a bias. You can have a "dummy" input (input(n+1) in your formualtion) which is always set to 1. Then the bias term is absorbed into the weights.</p>

CC BY-SA 3.0
2016-05-15T13:38:08.063Z
That's still a bias though. You're just shifting the work on the user instead of the internal logic of the system.
mtanti
CC BY-SA 3.0
2016-05-15T13:41:35.863Z
In that case, it seems that without a bias and using only "unshifted" sigmoid functions, you can only approximate functions that vanish at 0 -- i.e., not all functions.
Aryeh
CC BY-SA 3.0
2016-05-17T12:34:14.773Z
Do you know something that I can cite about this?
mtanti
CC BY-SA 3.0
2016-05-17T13:00:28.803Z
You can cite me :) -- more precisely, the simple claim that linear combinations and compositions of functions that vanish at zero must necessarily vanish at zero.
Aryeh
CC BY-SA 3.0
2016-05-17T13:01:24.590Z
As an expression of gratitude, you can mark my answer "correct".
Aryeh
CC BY-SA 3.0
2016-05-15T13:35:13.347Z
2016-05-15T13:35:13.347Z
Aryeh
<p>In a neural network, a bias is a constant term that is added to the weighted input in a neuron/unit:
output = activation_function( input1*weight1 + ... + inputn*weightn + bias)</p>

<p>I can see that the bias in a sigmoid activation function adds the ability to control the threshold of the activation. But when I started learning about neural nets we didn't use any bias, just the weighted inputs. I've also been told that sigmoid in particular can do without a bias, but this is not intuitively true for me at all.</p>

<p>So is it true that biases are redundant in sigmoid neural nets? How can neural nets learn to approximate any continuous function if they do not have a bias input?</p>

<machine-learning><ne.neural-evol>
Are biases necessary to make neural networks universal approximators when using sigmoid activations?
mtanti
# Are biases necessary to make neural networks universal approximators when using sigmoid activations?

(Asked by: username_0 on 2016-05-15)

In a neural network, a bias is a constant term that is added to the weighted input in a neuron/unit:
output = activation\_function( input1\*weight1 + ... + inputn\*weightn + bias)

I can see that the bias in a sigmoid activation function adds the ability to control the threshold of the activation. But when I started learning about neural nets we didn't use any bias, just the weighted inputs. I've also been told that sigmoid in particular can do without a bias, but this is not intuitively true for me at all.

So is it true that biases are redundant in sigmoid neural nets? How can neural nets learn to approximate any continuous function if they do not have a bias input?

**Tags:** machine-learning, ne.neural-evol

---

## 1 answers

### Answer by username_1 on 2016-05-15. Score: 4 | Accepted answer

No, you don't need a bias. You can have a "dummy" input (input(n+1) in your formualtion) which is always set to 1. Then the bias term is absorbed into the weights.

> **Comments:**
> **username_0**: That's still a bias though. You're just shifting the work on the user instead of the internal logic of the system.  
> **username_1**: In that case, it seems that without a bias and using only "unshifted" sigmoid functions, you can only approximate functions that vanish at 0 -- i.e., not all functions.  
> **username_0**: Do you know something that I can cite about this?  
> **username_1**: You can cite me :) -- more precisely, the simple claim that linear combinations and compositions of functions that vanish at zero must necessarily vanish at zero.  
> **username_1**: As an expression of gratitude, you can mark my answer "correct".  

---

