  Despite the practical success of deep neural networks, a comprehensive
theoretical framework that can predict practically relevant scores, such as the
test accuracy, from knowledge of the training data is currently lacking. Huge
simplifications arise in the infinite-width limit, where the number of units
$N_\ell$ in each hidden layer ($\ell=1,\dots, L$, being $L$ the depth of the
network) far exceeds the number $P$ of training examples. This idealisation,
however, blatantly departs from the reality of deep learning practice. Here, we
use the toolset of statistical mechanics to overcome these limitations and
derive an approximate partition function for fully-connected deep neural
architectures, which encodes information about the trained models. The
computation holds in the ''thermodynamic limit'' where both $N_\ell$ and $P$
are large and their ratio $\alpha_\ell = P/N_\ell$ is finite. This advance
allows us to obtain (i) a closed formula for the generalisation error
associated to a regression task in a one-hidden layer network with finite
$\alpha_1$; (ii) an approximate expression of the partition function for deep
architectures (via an ''effective action'' that depends on a finite number of
''order parameters''); (iii) a link between deep neural networks in the
proportional asymptotic limit and Student's $t$ processes.
