  The brain prepares for learning even before interacting with the environment,
by refining and optimizing its structures through spontaneous neural activity
that resembles random noise. However, the mechanism of such a process has yet
to be thoroughly understood, and it is unclear whether this process can benefit
the algorithm of machine learning. Here, we study this issue using a neural
network with a feedback alignment algorithm, demonstrating that pretraining
neural networks with random noise increases the learning efficiency as well as
generalization abilities without weight transport. First, we found that random
noise training modifies forward weights to match backward synaptic feedback,
which is necessary for teaching errors by feedback alignment. As a result, a
network with pre-aligned weights learns notably faster than a network without
random noise training, even reaching a convergence speed comparable to that of
a backpropagation algorithm. Sequential training with both random noise and
data brings weights closer to synaptic feedback than training solely with data,
enabling more precise credit assignment and faster learning. We also found that
each readout probability approaches the chance level and that the effective
dimensionality of weights decreases in a network pretrained with random noise.
This pre-regularization allows the network to learn simple solutions of a low
rank, reducing the generalization loss during subsequent training. This also
enables the network robustly to generalize a novel, out-of-distribution
dataset. Lastly, we confirmed that random noise pretraining reduces the amount
of meta-loss, enhancing the network ability to adapt to various tasks. Overall,
our results suggest that random noise training with feedback alignment offers a
straightforward yet effective method of pretraining that facilitates quick and
reliable learning without weight transport.
