  Just like weights, bias terms are the learnable parameters of many popular
machine learning models, including neural networks. Biases are thought to
enhance the representational power of neural networks, enabling them to solve a
variety of tasks in computer vision. However, we argue that biases can be
disregarded for some image-related tasks such as image classification, by
considering the intrinsic distribution of images in the input space and desired
model properties from first principles. Our findings suggest that zero-bias
neural networks can perform comparably to biased networks for practical image
classification tasks. We demonstrate that zero-bias neural networks possess a
valuable property called scalar (multiplication) invariance. This means that
the prediction of the network remains unchanged when the contrast of the input
image is altered. We extend scalar invariance to more general cases, enabling
formal verification of certain convex regions of the input space. Additionally,
we prove that zero-bias neural networks are fair in predicting the zero image.
Unlike state-of-the-art models that may exhibit bias toward certain labels,
zero-bias networks have uniform belief in all labels. We believe dropping bias
terms can be considered as a geometric prior in designing neural network
architecture for image classification, which shares the spirit of adapting
convolutions as the transnational invariance prior. The robustness and fairness
advantages of zero-bias neural networks may also indicate a promising path
towards trustworthy and ethical AI.
