  We introduce a novel computational unit for neural networks that features
multiple biases, challenging the traditional perceptron structure. This unit
emphasizes the importance of preserving uncorrupted information as it is passed
from one unit to the next, applying activation functions later in the process
with specialized biases for each unit. Through both empirical and theoretical
analyses, we show that by focusing on increasing biases rather than weights,
there is potential for significant enhancement in a neural network model's
performance. This approach offers an alternative perspective on optimizing
information flow within neural networks. See source code at
https://github.com/CuriosAI/dac-dev.
