  The nature of the energy source powering anomalous X-ray pulsars is
uncertain. Proposed scenarios involve either an ultramagnetized neutron star,
or accretion onto a neutron star. We consider the accretion model proposed
recently by Chatterjee, Hernquist & Narayan, in which a disk is fed by fallback
material following a supernova. We compute the optical, infrared, and
submillimeter emission expected from such a disk, including both viscous
dissipation and reradiation of X-ray flux impinging on the disk from the
pulsar. We find that it is possible with current instruments to put serious
constraints on this and on other accretion models of AXPs. Fallback disks could
also be found around isolated radio pulsars and we compute the corresponding
spectra. We show that the excess emission in the R and I bands observed for the
pulsar PSR 0656+14 is broadly consistent with emission from a disk.
