  The EDELWEISS-II collaboration has completed a direct search for WIMP dark
matter using cryogenic Ge detectors (400 g each) and 384 kg$\times$days of
effective exposure. A cross-section of $4.4 \times 10^{-8}$ pb is excluded at
90% C.L. for a WIMP mass of 85 GeV. The next phase, EDELWEISS-III, aims to
probe spin-independent WIMP-nucleon cross-sections down to a few
$\times10^{-9}$ pb. We present here the study of gamma and neutron background
coming from radioactive decays in the set-up and shielding materials. We have
carried out Monte Carlo simulations for the completed EDELWEISS-II setup with
GEANT4 and normalised the expected background rates to the measured
radioactivity levels (or their upper limits) of all materials and components.
The expected gamma-ray event rate in EDELWEISS-II at 20-200 keV agrees with the
observed rate of 82 events/kg/day within the uncertainties in the measured
concentrations. The calculated neutron rate from radioactivity of 1.0-3.1
events (90% C.L.) at 20-200 keV in the EDELWEISS-II data together with the
expected upper limit on the misidentified gamma-ray events ($\le0.9$), surface
betas ($\le0.3$), and muon-induced neutrons ($\le0.7$), do not contradict 5
observed events in nuclear recoil band. We have then extended the simulation
framework to the EDELWEISS-III configuration with 800 g crystals, better
material purity and additional neutron shielding inside the cryostat. The
gamma-ray and neutron backgrounds in 24 kg fiducial mass of EDELWEISS-III have
been calculated as 14-44 events/kg/day and 0.7-1.4 events per year,
respectively. The results of the background studies performed in the present
work have helped to select better purity components and improve shielding in
EDELWEISS-III to further reduce the expected rate of background events in the
next phase of the experiment.
