  Deep learning using Convolutional Neural Networks (CNNs) has been shown to
significantly out-performed many conventional vision algorithms. Despite
efforts to increase the CNN efficiency both algorithmically and with
specialized hardware, deep learning remains difficult to deploy in
resource-constrained environments. In this paper, we propose an end-to-end
framework to explore optically compute the CNNs in free-space, much like a
computational camera. Compared to existing free-space optics-based approaches
which are limited to processing single-channel (i.e., grayscale) inputs, we
propose the first general approach, based on nanoscale meta-surface optics,
that can process RGB data directly from the natural scenes. Our system achieves
up to an order of magnitude energy saving, simplifies the sensor design, all
the while sacrificing little network accuracy.
