  Designing algorithms for versatile AI hardware that can learn on the edge
using both labeled and unlabeled data is challenging. Deep end-to-end training
methods incorporating phases of self-supervised and supervised learning are
accurate and adaptable to input data but self-supervised learning requires even
more computational and memory resources than supervised learning, too high for
current embedded hardware. Conversely, unsupervised layer-by-layer training,
such as Hebbian learning, is more compatible with existing hardware but does
not integrate well with supervised learning. To address this, we propose a
method enabling networks or hardware designed for end-to-end supervised
learning to also perform high-performance unsupervised learning by adding two
simple elements to the output layer: Winner-Take-All (WTA) selectivity and
homeostasis regularization. These mechanisms introduce a "self-defined target"
for unlabeled data, allowing purely unsupervised training for both
fully-connected and convolutional layers using backpropagation or equilibrium
propagation on datasets like MNIST (up to 99.2%), Fashion-MNIST (up to 90.3%),
and SVHN (up to 81.5%). We extend this method to semi-supervised learning,
adjusting targets based on data type, achieving 96.6% accuracy with only 600
labeled MNIST samples in a multi-layer perceptron. Our results show that this
approach can effectively enable networks and hardware initially dedicated to
supervised learning to also perform unsupervised learning, adapting to varying
availability of labeled data.
