  We use mid-infrared Spitzer spectroscopy and far-infrared Herschel photometry
for a sample of twenty main sequence O9--B2 stars in the Small Magellanic Cloud
(SMC) with strong 24 micron excesses to investigate the origin of the mid-IR
emission. Either debris disks around the stars or illuminated patches of dense
interstellar medium (ISM) can cause such mid-IR emission. In a companion paper,
Paper I, we use optical spectroscopy to show that it is unlikely for any of
these sources to be classical Be stars or Herbig Ae/Be stars. We focus our
analysis on debris disks and cirrus hot spots. We find three out of twenty
stars to be significantly extended in the mid-IR, establishing them as cirrus
hot spots. We then fit the IR spectral energy distributions to determine dust
temperatures and masses. We find the dust masses in the SMC stars to be larger
than for any known debris disks, although this evidence against the debris disk
hypothesis is circumstantial. Finally, we created a local comparison sample of
bright mid-IR OB stars in the Milky Way (MW) by cross-matching the WISE and
Hipparcos catalogs. All such local stars in the appropriate luminosity range
that can be unambiguously classified are young stars with optical emission
lines or are spatially resolved by WISE with sizes too large to be plausible
debris disk candidates. We conclude that the very strong mid-IR flux excesses
are most likely explained as cirrus hot spots, although we cannot rigorously
rule out that a small fraction of the sample is made up of debris disks or
transition disks. We present suggestive evidence that bow-shock heating around
runaway stars may be a contributing mechanism to the interstellar emission.
These sources, interpreted as cirrus hot spots, offer a new localised probe of
diffuse interstellar dust in a low metallicity environment. (Abridged)
