<p>As mentioned by honi, it is possible to have bio plausible k-means. However, neither Leabra nor Hierarchical Temporal Memory (HTM) use it.</p>

<p>As Section 3.5.1 of How to Build a Brain by Chris Eliasmith notes:</p>

<blockquote>
  <p>There are basic computational methods underlying Leabra that are of
  dubious plausibility. The most evident is that Leabra directly applies
  a k-Winner-Takes-All (kWTA) algorithm, which is acknowledged as
  biologically implausible: “although the kWTA function is somewhat
  biologically implausible in its implementation (e.g., requiring global
  information about activation states and using sorting mechanisms), it
  provides a computationally effective approximation to biologically
  plausible inhibitory dynamics”
  (<a href="http://grey.colorado.edu/emergent/index.php/Leabra" rel="nofollow noreferrer">http://grey.colorado.edu/emergent/index.php/Leabra</a>; see also O’Reilly
  &amp; Munakata [2000], pp. 94-105 for further discussion). In other words,
  the actual dynamics of the system are replaced by an approximation
  that is computationally cheaper on digital computers.</p>
</blockquote>

<p>The same problem occurs with the HTM as discussed in <a href="https://forum.nengo.ai/t/hierarchical-temporal-memory-vs-nef-spa/257/12?u=seanny123" rel="nofollow noreferrer">this forum post by Travis DeWolf</a>:</p>

<blockquote>
  <p>In the second step of their spatial pooling, they
  find the k most active columns, to apply learning to only these
  columns. Dynamically, setting up WTA with lateral inhibitory
  connections is notoriously very tricky, and isn't something that can
  be done in a single time step. On top of that, controlling the
  learning so that it's only applied after the network has settled on a
  set of winners is a whole other issue. It might be the case it works
  running the whole time as the WTA circuit settles, but the dynamics
  are complex and can't just be assumed to work.</p>
</blockquote>

<p>To summarize, k-winner inhibition is possible, but it's not used in a biologically plausible manner in either Leabra and HTM.</p>

CC BY-SA 3.0
2017-05-28T12:41:57.447Z
2017-05-28T12:41:57.447Z
Seanny123
<p>Yes, see "<a href="http://www.jneurosci.org/content/29/23/7497.short" rel="nofollow noreferrer">A Second Function of Gamma Frequency Oscillations: An E%-Max Winner-Take-All Mechanism Selects Which Cells Fire</a>" by de Almeida, et al., 2009 for a biologically plausible implementation.</p>

CC BY-SA 3.0
2016-01-28T17:46:43.777Z
2017-05-28T10:37:56.847Z
2017-05-28T10:37:56.847Z
honi
<p>In the Leabra cognitive architecture, sparse representations are created by simply ignoring all but the k strongest activations, if I understand correctly. In Hierarchical Temporal Memory, instead, each firing neuron inhibits an area of a given radius during this time step. To me, this seems more plausible due to the locality of neuronal connections.</p>

<p>Is the k-winner inhibition biologically plausible?</p>

<neuroscience><cognitive-modeling>
Is k-winner a biological plausible model of inhibition?
danijar
# Is k-winner a biological plausible model of inhibition?

(Asked by: username_0 on 2016-01-28)

In the Leabra cognitive architecture, sparse representations are created by simply ignoring all but the k strongest activations, if I understand correctly. In Hierarchical Temporal Memory, instead, each firing neuron inhibits an area of a given radius during this time step. To me, this seems more plausible due to the locality of neuronal connections.

Is the k-winner inhibition biologically plausible?

**Tags:** neuroscience, cognitive-modeling


---

## 2 answers

### Answer by username_1 on 2017-05-28. Score: 3 | Accepted answer

As mentioned by username_2, it is possible to have bio plausible k-means. However, neither Leabra nor Hierarchical Temporal Memory (HTM) use it.

As Section 3.5.1 of How to Build a Brain by Chris Eliasmith notes:

> There are basic computational methods underlying Leabra that are of
> dubious plausibility. The most evident is that Leabra directly applies
> a k-Winner-Takes-All (kWTA) algorithm, which is acknowledged as
> biologically implausible: “although the kWTA function is somewhat
> biologically implausible in its implementation (e.g., requiring global
> information about activation states and using sorting mechanisms), it
> provides a computationally effective approximation to biologically
> plausible inhibitory dynamics”
> (<http://grey.colorado.edu/emergent/index.php/Leabra>; see also O’Reilly
> & Munakata [2000], pp. 94-105 for further discussion). In other words,
> the actual dynamics of the system are replaced by an approximation
> that is computationally cheaper on digital computers.

The same problem occurs with the HTM as discussed in [this forum post by Travis DeWolf](https://forum.nengo.ai/t/hierarchical-temporal-memory-vs-nef-spa/257/12?u=seanny123):

> In the second step of their spatial pooling, they
> find the k most active columns, to apply learning to only these
> columns. Dynamically, setting up WTA with lateral inhibitory
> connections is notoriously very tricky, and isn't something that can
> be done in a single time step. On top of that, controlling the
> learning so that it's only applied after the network has settled on a
> set of winners is a whole other issue. It might be the case it works
> running the whole time as the WTA circuit settles, but the dynamics
> are complex and can't just be assumed to work.

To summarize, k-winner inhibition is possible, but it's not used in a biologically plausible manner in either Leabra and HTM.


---

### Answer by username_2 on 2016-01-28. Score: 2

Yes, see "[A Second Function of Gamma Frequency Oscillations: An E%-Max Winner-Take-All Mechanism Selects Which Cells Fire](http://www.jneurosci.org/content/29/23/7497.short)" by de Almeida, et al., 2009 for a biologically plausible implementation.


---

