  Neural computation in biological and artificial networks relies on the
nonlinear summation of many inputs. The structural connectivity matrix of
synaptic weights between neurons is a critical determinant of overall network
function, but quantitative links between neural network structure and function
are complex and subtle. For example, many networks can give rise to similar
functional responses, and the same network can function differently depending
on context. Whether certain patterns of synaptic connectivity are required to
generate specific network-level computations is largely unknown. Here we
introduce a geometric framework for identifying synaptic connections required
by steady-state responses in recurrent networks of threshold-linear neurons.
Assuming that the number of specified response patterns does not exceed the
number of input synapses, we analytically calculate the solution space of all
feedforward and recurrent connectivity matrices that can generate the specified
responses from the network inputs. A generalization accounting for noise
further reveals that the solution space geometry can undergo topological
transitions as the allowed error increases, which could provide insight into
both neuroscience and machine learning. We ultimately use this geometric
characterization to derive certainty conditions guaranteeing a non-zero synapse
between neurons. Our theoretical framework could thus be applied to neural
activity data to make rigorous anatomical predictions that follow generally
from the model architecture.
