  Spatial Fluctuations in the Cosmic Infrared Background have now been measured
out to sub-degree scales showing a strong clustering signal from unresolved
sources. We attempt to explain these measurement by considering faint galaxy
populations at z<6 as the underlying sources for this signal using 233 measured
UV, optical and NIR luminosity functions (LF) from a variety of surveys
covering a wide range of redshifts. We populate the lightcone and calculate the
total emission redshifted into the near-IR bands in the observer frame and
recover the observed optical and near-IR galaxy counts to a good accuracy.
Using a halo model for the clustering of galaxies with an underlying LCDM
density field, we find that fluctuations from known galaxy populations are
unable to account for the large scale CIB clustering signal seen by HST/NICMOS,
Spitzer/IRAC and AKARI/IRC and continue to diverge out to larger angular
scales. Our purely empirical reconstruction shows that known galaxy populations
are not responsible for the bulk of the fluctuation signal seen in the
measurements and suggests an unknown population of very faint and highly
clustered sources dominating the signal.
