  Targeting vision applications at the edge, in this work, we systematically
explore and propose a high-performance and energy-efficient Optical In-Sensor
Accelerator architecture called OISA for the first time. Taking advantage of
the promising efficiency of photonic devices, the OISA intrinsically implements
a coarse-grained convolution operation on the input frames in an innovative
minimum-conversion fashion in low-bit-width neural networks. Such a design
remarkably reduces the power consumption of data conversion, transmission, and
processing in the conventional cloud-centric architecture as well as
recently-presented edge accelerators. Our device-to-architecture simulation
results on various image data-sets demonstrate acceptable accuracy while OISA
achieves 6.68 TOp/s/W efficiency. OISA reduces power consumption by a factor of
7.9 and 18.4 on average compared with existing electronic in-/near-sensor and
ASIC accelerators.
