  Some models of inelastic dark matter posit the existence of bound states
under some new $U(1)'$ gauge symmetry. If this new dark photon kinetically
mixes with the standard model photon, then the constituent particles in these
bound states can acquire a fractional electric charge. This electric charge
renders a dark-matter medium dispersive. We compute this frequency-dependent
index of refraction for such a medium and use the frequency-dependent arrival
time of light from astrophysical sources to constrain the properties of dark
atoms in the medium. Using optical-wavelength observations from the Crab
Pulsar, we find the electric millicharge of dark (electrons) protons to be
smaller than the electric charge $e$ for dark atom masses below 100 keV,
assuming a dark fine structure constant $\boldsymbol{\alpha}=1$. We estimate
that future broadband observations of gamma-ray bursts can produce constraints
on the millicharge of dark atoms with masses in the keV range that are
competitive with existing collider constraints.
