  We present a novel method of CNN inference for pixel processor array (PPA)
vision sensors, designed to take advantage of their massive parallelism and
analog compute capabilities. PPA sensors consist of an array of processing
elements (PEs), with each PE capable of light capture, data storage and
computation, allowing various computer vision processing to be executed
directly upon the sensor device. The key idea behind our approach is storing
network weights "in-pixel" within the PEs of the PPA sensor itself to allow
various computations, such as multiple different image convolutions, to be
carried out in parallel. Our approach can perform convolutional layers, max
pooling, ReLu, and a final fully connected layer entirely upon the PPA sensor,
while leaving no untapped computational resources. This is in contrast to
previous works that only use a sensor-level processing to sequentially compute
image convolutions, and must transfer data to an external digital processor to
complete the computation. We demonstrate our approach on the SCAMP-5 vision
system, performing inference of a MNIST digit classification network at over
3000 frames per second and over 93% classification accuracy. This is the first
work demonstrating CNN inference conducted entirely upon the processor array of
a PPA vision sensor device, requiring no external processing.
