  The Forward Forward algorithm, proposed by Geoffrey Hinton in November 2022,
is a novel method for training neural networks as an alternative to
backpropagation. In this project, we replicate Hinton's experiments on the
MNIST dataset, and subsequently extend the scope of the method with two
significant contributions. First, we establish a baseline performance for the
Forward Forward network on the IMDb movie reviews dataset. As far as we know,
our results on this sentiment analysis task marks the first instance of the
algorithm's extension beyond computer vision. Second, we introduce a novel
pyramidal optimization strategy for the loss threshold - a hyperparameter
specific to the Forward Forward method. Our pyramidal approach shows that a
good thresholding strategy causes a difference of up to 8% in test error.
Lastly, we perform visualizations of the trained parameters and derived several
significant insights, such as a notably larger (10-20x) mean and variance in
the weights acquired by the Forward Forward network.
  Repository: https://github.com/Ads-cmu/ForwardForward
