  Networks of neurons in the brain encode preferred patterns of neural activity
via their synaptic connections. Despite receiving considerable attention, the
precise relationship between network connectivity and encoded patterns is still
poorly understood. Here we consider this problem for networks of
threshold-linear neurons whose computational function is to learn and store a
set of binary patterns (e.g., a neural code) as "permitted sets" of the
network. We introduce a simple Encoding Rule that selectively turns "on"
synapses between neurons that co-appear in one or more patterns. The rule uses
synapses that are binary, in the sense of having only two states ("on" or
"off"), but also heterogeneous, with weights drawn from an underlying synaptic
strength matrix S. Our main results precisely describe the stored patterns that
result from the Encoding Rule -- including unintended "spurious" states -- and
give an explicit characterization of the dependence on S. In particular, we
find that binary patterns are successfully stored in these networks when the
excitatory connections between neurons are geometrically balanced -- i.e., they
satisfy a set of geometric constraints. Furthermore, we find that certain types
of neural codes are "natural" in the context of these networks, meaning that
the full code can be accurately learned from a highly undersampled set of
patterns. Interestingly, many commonly observed neural codes in cortical and
hippocampal areas are natural in this sense. As an application, we construct
networks that encode hippocampal place field codes nearly exactly, following
presentation of only a small fraction of patterns. To obtain our results, we
prove new theorems using classical ideas from convex and distance geometry,
such as Cayley-Menger determinants, revealing a novel connection between these
areas of mathematics and coding properties of neural networks.
