  The paper aims at removing the aliasing effects of the whole focal stack
generated from a sparse-sampled {4D} light field, while keeping the consistency
across all the focal layers. We first explore the structural characteristics
embedded in the focal stack slice and its corresponding frequency-domain
representation, i.e., the Focal Stack Spectrum (FSS). We observe that the
energy distribution of the FSS always resides within the same triangular area
under different angular sampling rates, additionally the continuity of the
Point Spread Function (PSF) is intrinsically maintained in the FSS. Based on
these two observations, we propose a learning-based FSS reconstruction approach
for one-time aliasing removing over the whole focal stack. Moreover, a novel
conjugate-symmetric loss function is proposed for the optimization. Compared to
previous works, our method avoids an explicit depth estimation, and can handle
challenging large-disparity scenarios. Experimental results on both synthetic
and real light field datasets show the superiority of the proposed approach for
different scenes and various angular sampling rates.
