  The Backpropagation algorithm has often been criticised for its lack of
biological realism. In an attempt to find a more biologically plausible
alternative, the recently introduced Forward-Forward algorithm replaces the
forward and backward passes of Backpropagation with two forward passes. In this
work, we show that the internal representations obtained by the Forward-Forward
algorithm can organise into category-specific ensembles exhibiting high
sparsity - composed of a low number of active units. This situation is
reminiscent of what has been observed in cortical sensory areas, where neuronal
ensembles are suggested to serve as the functional building blocks for
perception and action. Interestingly, while this sparse pattern does not
typically arise in models trained with standard Backpropagation, it can emerge
in networks trained with Backpropagation on the same objective proposed for the
Forward-Forward algorithm. These results suggest that the learning procedure
proposed by Forward-Forward may be superior to Backpropagation in modelling
learning in the cortex, even when a backward pass is used.
