  A high efficiency hardware integration of neural networks benefits from
realizing nonlinearity, network connectivity and learning fully in a physical
substrate. Multiple systems have recently implemented some or all of these
operations, yet the focus was placed on addressing technological challenges.
Fundamental questions regarding learning in hardware neural networks remain
largely unexplored. Noise in particular is unavoidable in such architectures,
and here we investigate its interaction with a learning algorithm using an
opto-electronic recurrent neural network. We find that noise strongly modifies
the system's path during convergence, and surprisingly fully decorrelates the
final readout weight matrices. This highlights the importance of understanding
architecture, noise and learning algorithm as interacting players, and
therefore identifies the need for mathematical tools for noisy, analogue system
optimization.
