  Motivated by the idea that criticality and universality of phase transitions
might play a crucial role in achieving and sustaining learning and intelligent
behaviour in biological and artificial networks, we analyse a theoretical and a
pragmatic experimental set up for critical phenomena in deep learning. On the
theoretical side, we use results from statistical physics to carry out critical
point calculations in feed-forward/fully connected networks, while on the
experimental side we set out to find traces of criticality in deep neural
networks. This is our first step in a series of upcoming investigations to map
out the relationship between criticality and learning in deep networks.
