  Semi-supervised learning algorithms typically construct a weighted graph of
data points to represent a manifold. However, an explicit graph representation
is problematic for neural networks operating in the online setting. Here, we
propose a feed-forward neural network capable of semi-supervised learning on
manifolds without using an explicit graph representation. Our algorithm uses
channels that represent localities on the manifold such that correlations
between channels represent manifold structure. The proposed neural network has
two layers. The first layer learns to build a representation of low-dimensional
manifolds in the input data as proposed recently in [8]. The second learns to
classify data using both occasional supervision and similarity of the manifold
representation of the data. The channel carrying label information for the
second layer is assumed to be "silent" most of the time. Learning in both
layers is Hebbian, making our network design biologically plausible. We
experimentally demonstrate the effect of semi-supervised learning on
non-trivial manifolds.
