  Advancing our knowledge of how the brain processes information remains a key
challenge in neuroscience. This thesis combines three different approaches to
the study of the dynamics of neural networks and their encoding
representations: a computational approach, that builds upon basic biological
features of neurons and their networks to construct effective models that can
simulate their structure and dynamics; a machine-learning approach, which draws
a parallel with the functional capabilities of brain networks, allowing us to
infer the dynamical and encoding properties required to solve certain
input-processing tasks; and a final, theoretical treatment, which will take us
into the fascinating hypothesis of the "critical" brain as the mathematical
foundation that can explain the emergent collective properties arising from the
interactions of millions of neurons. Hand in hand with physics, we venture into
the realm of neuroscience to explain the existence of quasi-universal scaling
properties across brain regions, setting out to quantify the distance of their
dynamics from a critical point. Next, we move into the grounds of artificial
intelligence, where the very same theory of critical phenomena will prove very
useful for explaining the effects of biologically-inspired plasticity rules in
the forecasting ability of Reservoir Computers. Halfway into our journey, we
explore the concept of neural representations of external stimuli, unveiling a
surprising link between the dynamical regime of neural networks and the optimal
topological properties of such representation manifolds. The thesis ends with
the singular problem of representational drift in the process of odor encoding
carried out by the olfactory cortex, uncovering the potential synaptic
plasticity mechanisms that could explain this recently observed phenomenon.
