  Holographic displays can generate light fields by dynamically modulating the
wavefront of a coherent beam of light using a spatial light modulator,
promising rich virtual and augmented reality applications. However, the limited
spatial resolution of existing dynamic spatial light modulators imposes a tight
bound on the diffraction angle. As a result, modern holographic displays
possess low \'{e}tendue, which is the product of the display area and the
maximum solid angle of diffracted light. The low \'{e}tendue forces a sacrifice
of either the field-of-view (FOV) or the display size. In this work, we lift
this limitation by presenting neural \'{e}tendue expanders. This new breed of
optical elements, which is learned from a natural image dataset, enables higher
diffraction angles for ultra-wide FOV while maintaining both a compact form
factor and the fidelity of displayed contents to human viewers. With neural
\'{e}tendue expanders, we experimentally achieve 64$\times$ \'{e}tendue
expansion of natural images in full color, expanding the FOV by an order of
magnitude horizontally and vertically, with high-fidelity reconstruction
quality (measured in PSNR) over 29 dB on retinal-resolution images.
