  In this paper, we study recurrent neural networks in the presence of pairwise
learning rules. We are specifically interested in how the attractor landscapes
of such networks become altered as a function of the strength and nature
(Hebbian vs. anti-Hebbian) of learning, which may have a bearing on the ability
of such rules to mediate large-scale optimization problems. Through formal
analysis, we show that a transition from Hebbian to anti-Hebbian learning
brings about a pitchfork bifurcation that destroys convexity in the network
attractor landscape. In larger-scale settings, this implies that anti-Hebbian
plasticity will bring about multiple stable equilibria, and such effects may be
outsized at interconnection or `choke' points. Furthermore, attractor
landscapes are more sensitive to slower learning rates than faster ones. These
results provide insight into the types of objective functions that can be
encoded via different pairwise plasticity rules.
