  We present J and Ks band galaxy counts down to J=24 and Ks=22.5 obtained with
the new infrared imager/spectrometer, SOFI, at the ESO New Technology
Telescope. The co-addition of short, dithered, images led to a total exposure
time of 256 and 624 minutes respectively, over an area of $\sim20$ arcmin$^2$
centered on the NTT Deep Field. The total number of sources with S/N$>5$ is
1569 in the J sample and 1025 in the Ks-selected sample. These are the largest
samples currently available at these depths. A d$logN$/d$m$ relation with slope
of $\sim0.36$ in J and $\sim0.38$ in Ks is found with no evident sign of a
decline at the magnitude limit. The observed surface density of ``small''
sources is much lower than ``large'' ones at bright magnitudes and rises more
steeply than the large sources to fainter magnitudes. Fainter than $J\sim22.5$
and Ks$\sim21.5$, small sources dominate the number counts. Galaxies get redder
in J-K down to
  J$\sim20$ and Ks$\sim19$. At fainter magnitudes, the median color becomes
bluer with an accompanying increase in the compactness of the galaxies. We show
that the blue, small sources which dominate the faint IR counts are not
compatible with a high redshift ($z>1$) population. On the contrary, the
observed color and compactness trends, together with the absence of a turnover
at faint magnitudes and the dominance of small sources, can be naturally
explained by an increasing contribution of sub-$L^*$ galaxies when going to
fainter apparent magnitudes. Such evidence strongly supports the existence of a
steeply rising ($\alpha\ll-1$) faint end of the local infrared luminosity
function of galaxies - at least for luminosities $L<0.01L^*$.
