  The rapid advancements in machine learning across numerous industries have
amplified the demand for extensive matrix-vector multiplication operations,
thereby challenging the capacities of traditional von Neumann computing
architectures. To address this, researchers are currently exploring
alternatives such as in-memory computing systems to develop faster and more
energy-efficient hardware. In particular, there is renewed interest in
computing systems based on optics, which could potentially handle matrix-vector
multiplication in a more energy-efficient way. Despite promising initial
results, developing a highly parallel, programmable, and scalable optical
computing system capable of rivaling electronic computing hardware still
remains elusive. In this context, we propose a hyperspectral in-memory
computing architecture that integrates space multiplexing with frequency
multiplexing of optical frequency combs and uses spatial light modulators as a
programmable optical memory, thereby boosting the computational throughput and
the energy efficiency. We have experimentally demonstrated multiply-accumulate
operations with higher than 4-bit precision in both matrix-vector and
matrix-matrix multiplications, which suggests the system's potential for a wide
variety of deep learning and optimization tasks. This system exhibits
extraordinary modularity, scalability, and programmability, effectively
transcending the traditional limitations of optics-based computing
architectures. Our approach demonstrates the potential to scale beyond peta
operations per second, marking a significant step towards achieving
high-throughput energy-efficient optical computing.
