  Hybrid refractive-diffractive lenses combine the light efficiency of
refractive lenses with the information encoding power of diffractive optical
elements (DOE), showing great potential as the next generation of imaging
systems. However, accurately simulating such hybrid designs is generally
difficult, and in particular, there are no existing differentiable image
formation models for hybrid lenses with sufficient accuracy.
  In this work, we propose a new hybrid ray-tracing and wave-propagation
(ray-wave) model for accurate simulation of both optical aberrations and
diffractive phase modulation, where the DOE is placed between the last
refractive surface and the image sensor, i.e. away from the Fourier plane that
is often used as a DOE position. The proposed ray-wave model is fully
differentiable, enabling gradient back-propagation for end-to-end co-design of
refractive-diffractive lens optimization and the image reconstruction network.
We validate the accuracy of the proposed model by comparing the simulated point
spread functions (PSFs) with theoretical results, as well as simulation
experiments that show our model to be more accurate than solutions implemented
in commercial software packages like Zemax. We demonstrate the effectiveness of
the proposed model through real-world experiments and show significant
improvements in both aberration correction and extended depth-of-field (EDoF)
imaging. We believe the proposed model will motivate further investigation into
a wide range of applications in computational imaging, computational
photography, and advanced optical design. Code will be released upon
publication.
