<p>As explained in Denis Cousineau's answer, there are many recurrent neural networks who's activities can be described as attractors. However, I would like to highlight two cases in particular, the Neural Engineering Framework, Conceptors and FORCE trained networks.</p>

<p>In the Neural Engineering Framework case, a recurrent neural network can approximate any dynamical system, including chaotic attractors. <a href="https://github.com/celiasmith/syde556/blob/master/SYDE%20556%20Lecture%205%20Dynamics.ipynb" rel="nofollow noreferrer">See these notes for a derivation and a demonstration</a>. The networks can achieve certain stable states, for example when emulating a memory.</p>

<p><a href="https://arxiv.org/abs/1403.3369" rel="nofollow noreferrer">Conceptors</a> are similar to the Neural Engineering Framework, except they questionably swap out recurrent weights to generate different patterns.</p>

<p>In <a href="https://arxiv.org/abs/1609.02545" rel="nofollow noreferrer">FORCE training</a>, a chaotically spiking reservoir can be trained to approximate seemingly any desired dynamical system.</p>

<p>In all these cases, assuming you have a system that can be seen as a transition between various stable states, I don't see why each state wouldn't be possible to map onto the node of a graph.</p>

CC BY-SA 3.0
2017-01-30T05:13:25.873Z
2017-01-30T05:13:25.873Z
Seanny123
<p><strong>Only a nonlinear recurrent system can have interesting attractors</strong>: i.e., feedforward networks cannot have any attractor, and linear dynamical system can only have one (stable) fixed point (i.e., point attractor).</p>

<p>The relevant area of math is <a href="https://en.wikipedia.org/wiki/Dynamical_systems_theory" rel="nofollow noreferrer">dynamical systems theory</a>.</p>

CC BY-SA 3.0
2017-02-05T08:41:10.807Z
Can you please elaborate on the connection between the chaos's attractor and the one context of neuroscience?
0x90
CC BY-SA 3.0
2017-02-05T11:22:26.817Z
@0x90 I think that should be a separate question.
Memming
CC BY-SA 3.0
2017-02-04T22:53:04.210Z
2017-02-04T22:53:04.210Z
Memming
<p>Many recurrent networks have attractors. For example, the BAM (bidirectional associative network) is a recurrent network: one input is presented and the network cycles until no more changes are seen in the connection weights. The BAM can be understood as developing attractor states: Seen in n-dimensions (in which n is the number of units in the model), the state can be seen as a position. Whenever the same stimuli is presented, the state of the network will stabilize to the same position. </p>

<p>One reference is Chartier S, Boukadoum M. (2006) A bidirectional heteroassociative memory for binary and grey-level patterns. IEEE Transactions on Neural Networks, 17(2):385-96.</p>

CC BY-SA 3.0
2017-01-29T22:33:43.637Z
2017-02-04T17:50:10.930Z
2017-02-04T17:50:10.930Z
Denis Cousineau
<p>How are attractor and recurrent networks related? Is attractor network is <a href="https://en.wikipedia.org/wiki/Limiting_case_(mathematics)" rel="nofollow noreferrer">private case</a> of recurrent? Is there a mathematical formalism that defines both rigorously as <a href="https://en.wikipedia.org/wiki/Graph_(discrete_mathematics)" rel="nofollow noreferrer">graphs</a>? </p>

<neural-network>
What is the difference between attractor and recurrent network?
0x90
# What is the difference between attractor and recurrent network?

(Asked by: username_0 on 2017-01-27)

How are attractor and recurrent networks related? Is attractor network is [private case](https://en.wikipedia.org/wiki/Limiting_case_(mathematics)) of recurrent? Is there a mathematical formalism that defines both rigorously as [graphs](https://en.wikipedia.org/wiki/Graph_(discrete_mathematics))?

**Tags:** neural-network


---

## 3 answers

### Answer by username_3 on 2017-01-29. Score: 4 | Accepted answer

Many recurrent networks have attractors. For example, the BAM (bidirectional associative network) is a recurrent network: one input is presented and the network cycles until no more changes are seen in the connection weights. The BAM can be understood as developing attractor states: Seen in n-dimensions (in which n is the number of units in the model), the state can be seen as a position. Whenever the same stimuli is presented, the state of the network will stabilize to the same position.

One reference is Chartier S, Boukadoum M. (2006) A bidirectional heteroassociative memory for binary and grey-level patterns. IEEE Transactions on Neural Networks, 17(2):385-96.


---

### Answer by username_1 on 2017-01-30. Score: 3

As explained in username_3's answer, there are many recurrent neural networks who's activities can be described as attractors. However, I would like to highlight two cases in particular, the Neural Engineering Framework, Conceptors and FORCE trained networks.

In the Neural Engineering Framework case, a recurrent neural network can approximate any dynamical system, including chaotic attractors. [See these notes for a derivation and a demonstration](https://github.com/celiasmith/syde556/blob/master/SYDE%20556%20Lecture%205%20Dynamics.ipynb). The networks can achieve certain stable states, for example when emulating a memory.

[Conceptors](https://arxiv.org/abs/1403.3369) are similar to the Neural Engineering Framework, except they questionably swap out recurrent weights to generate different patterns.

In [FORCE training](https://arxiv.org/abs/1609.02545), a chaotically spiking reservoir can be trained to approximate seemingly any desired dynamical system.

In all these cases, assuming you have a system that can be seen as a transition between various stable states, I don't see why each state wouldn't be possible to map onto the node of a graph.


---

### Answer by username_2 on 2017-02-04. Score: 1

**Only a nonlinear recurrent system can have interesting attractors**: i.e., feedforward networks cannot have any attractor, and linear dynamical system can only have one (stable) fixed point (i.e., point attractor).

The relevant area of math is [dynamical systems theory](https://en.wikipedia.org/wiki/Dynamical_systems_theory).

> **Comments:**
> **username_0**: Can you please elaborate on the connection between the chaos's attractor and the one context of neuroscience?  
> **username_2**: @username_0 I think that should be a separate question.  

---

