  We present a novel deep learning approach to reconstruct confocal microscopy
stacks from single light field images. To perform the reconstruction, we
introduce the LFMNet, a novel neural network architecture inspired by the U-Net
design. It is able to reconstruct with high-accuracy a 112x112x57.6$\mu m^3$
volume (1287x1287x64 voxels) in 50ms given a single light field image of
1287x1287 pixels, thus dramatically reducing 720-fold the time for confocal
scanning of assays at the same volumetric resolution and 64-fold the required
storage. To prove the applicability in life sciences, our approach is evaluated
both quantitatively and qualitatively on mouse brain slices with fluorescently
labelled blood vessels. Because of the drastic reduction in scan time and
storage space, our setup and method are directly applicable to real-time in
vivo 3D microscopy. We provide analysis of the optical design, of the network
architecture and of our training procedure to optimally reconstruct volumes for
a given target depth range. To train our network, we built a data set of 362
light field images of mouse brain blood vessels and the corresponding aligned
set of 3D confocal scans, which we use as ground truth. The data set will be
made available for research purposes.
