  Several independent measurements have confirmed the existence of fluctuations
($\delta F_{\rm obs}\approx 0.1 \rm nW/m^{2}/sr$ at $3.6 \rm \mu m$) up to
degree angular scales in the source-subtracted Near InfraRed Background (NIRB)
whose origin is unknown. By combining high resolution cosmological
N-body/hydrodynamical simulations with an analytical model, and by matching
galaxy Luminosity Functions (LFs) and the constraints on reionization
simultaneously, we predict the NIRB absolute flux and fluctuation amplitude
produced by high-$z$ ($z > 5$) galaxies (some of which harboring Pop III stars,
shown to provide a negligible contribution). This strategy also allows us to
make an empirical determination of the evolution of ionizing photon escape
fraction: we find $f_{\rm esc} = 1$ at $z \ge 11$, decreasing to $\approx 0.05$
at $z = 5$. In the wavelength range $1.0-4.5 \rm \mu m$, the predicted
cumulative flux is $F =0.2-0.04 \rm nW/m^2/sr$. However, we find that the
radiation from high-$z$ galaxies (including those undetected by current
surveys) is insufficient to explain the amplitude of the observed fluctuations:
at $l=2000$, the fluctuation level due to $z > 5$ galaxies is $\delta F =
0.01-0.002 \rm nW/m^2/sr$, with a relative wavelength-independent amplitude
$\delta F/F = 4$%. The source of the missing power remains unknown. This might
indicate that an unknown component/foreground, with a clustering signal very
similar to that of high-$z$ galaxies, dominates the source-subtracted NIRB
fluctuation signal.
