  Ambient neutrons are one of the most serious backgrounds for underground
experiments searching for rare events. The ambient neutron flux in an
underground laboratory of Kamioka Observatory was measured using a
$\mathrm{^3He}$ proportional counter with various moderator setups. Since the
detector response largely depends on the spectral shape, the energy spectra of
the neutrons transported from the rock to the laboratory are estimated by
Monte-Carlo simulations. The ratio of the thermal neutron flux to the total
neutron flux was found to depend on the thermalizing efficiency of the rock.
Therefore, the ratio of the count rate without a moderator to that with a
moderator was used to determine this parameter. Consequently, the most-likely
neutron spectrum predicted by the simulations for the parameters determined by
the experimental results was obtained. The result suggests an interesting
spectral shape, which has not been indicated in previous studies. The total
ambient neutron flux is $(23.52 \pm 0.68 \ \mathrm{_{stat.}} ^{+1.87}_{-2.13} \
\mathrm{_{sys.}}) \times 10^{-6}$ cm$^{-2}$ s$^{-1}$. This result, especially
the energy spectrum information, could be a new and important input for
estimating the background in current and future experiments in the underground
laboratory at Kamioka Observatory.
