  The architecture of a neural network constrains the potential dynamics that
can emerge. Some architectures may only allow for a single dynamic regime,
while others display a great deal of flexibility with qualitatively different
dynamics that can be reached by modulating connection strengths. In this work,
we develop novel mathematical techniques to study the dynamic constraints
imposed by different network architectures in the context of competitive
threshold-linear networks (TLNs). Any given TLN is naturally characterized by a
hyperplane arrangement in $\mathbb{R}^n$, and the combinatorial properties of
this arrangement determine the pattern of fixed points of the dynamics. This
observation enables us to recast the question of network flexibility in the
language of oriented matroids, allowing us to employ tools and results from
this theory in order to characterize the different dynamic regimes a given
architecture can support. In particular, fixed points of a TLN correspond to
cocircuits of an associated oriented matroid; and mutations of the matroid
correspond to bifurcations in the collection of fixed points. As an
application, we provide a complete characterization of all possible sets of
fixed points that can arise in networks through size $n=3$, together with
descriptions of how to modulate synaptic strengths of the network in order to
access the different dynamic regimes. These results provide a framework for
studying the possible computational roles of various motifs observed in real
neural networks.
