  The optical conductivity of MgB$_2$ has been determined on a dense
polycrystalline sample in the spectral range 6 meV - 4.6 eV using a combination
of ellipsometric and normal incidence reflectivity measurements.
$\sigma_{1}(\omega)$ features a narrow Drude peak with anomalously small plasma
frequency (1.4 eV) and a very broad "dome" structure, which comprises the bulk
of the low-energy spectral weight. This fact can be reconciled with the results
of band structure calculations by assuming that charge carriers from the 2D
$\sigma$-bands and the 3D $\pi$-bands have principally different impurity
scattering rates and negligible interband scattering. This also explains a
surprisingly small correlation between the defect concentration and $T_c$,
expected for a two-gap superconductor. The large 3D carrier scattering rate
suggests their proximity to the localization limit.
