  In recent years, machine vision has taken huge leaps and is now becoming an
integral part of various intelligent systems, including autonomous vehicles,
robotics, and many others. Usually, visual information is captured by a
frame-based camera, converted into a digital format, and processed afterwards
using a machine learning algorithm such as an artificial neural network (ANN).
A large amount of (mostly redundant) data being passed through the entire
signal chain, however, results in low frame rates and large power consumption.
Various visual data preprocessing techniques have thus been developed that
allow to increase the efficiency of the subsequent signal processing in an ANN.
Here, we demonstrate that an image sensor itself can constitute an ANN that is
able to simultaneously sense and process optical images without latency. Our
device is based on a reconfigurable two-dimensional (2D) semiconductor
photodiode array, with the synaptic weights of the network being stored in a
continuously tunable photoresponsivity matrix. We demonstrate both supervised
and unsupervised learning and successfully train the sensor to classify and
encode images, that are optically projected onto the chip, with a throughput of
20 million bins per second.
