  Neural-network architectures have been increasingly used to represent quantum
many-body wave functions. These networks require a large number of variational
parameters and are challenging to optimize using traditional methods, as
gradient descent. Stochastic Reconfiguration (SR) has been effective with a
limited number of parameters, but becomes impractical beyond a few thousand
parameters. Here, we leverage a simple linear algebra identity to show that SR
can be employed even in the deep learning scenario. We demonstrate the
effectiveness of our method by optimizing a Deep Transformer architecture with
$3 \times 10^5$ parameters, achieving state-of-the-art ground-state energy in
the $J_1$-$J_2$ Heisenberg model at $J_2/J_1=0.5$ on the $10\times10$ square
lattice, a challenging benchmark in highly-frustrated magnetism. This work
marks a significant step forward in the scalability and efficiency of SR for
Neural-Network Quantum States, making them a promising method to investigate
unknown quantum phases of matter, where other methods struggle.
