  Gravitational encounters between small-scale dark matter substructure and
cold stellar streams in the Milky Way halo lead to density perturbations in the
latter, making streams an effective probe for detecting dark matter
substructure. The Pal 5 stream is one such system for which we have some of the
best data. However, Pal 5 orbits close to the center of the Milky Way and has
passed through the Galactic disk many times, where its structure can be
perturbed by baryonic structures such as the Galactic bar and giant molecular
clouds (GMCs). In order to understand how these baryonic structures affect Pal
5's density, we present a detailed study of the effects of the Galactic bar,
spiral structure, GMCs, and globular clusters on the Pal 5 stream. We estimate
the effect of each perturber on the stream density by computing its power
spectrum and comparing it to the power induced by a CDM-like population of dark
matter subhalos. We find that the bar and GMCs can each individually create
power that is comparable to the observed power on large scales, leaving little
room for dark matter substructure, while spirals are subdominant on all scales.
On degree scales, the power induced by the bar is small, but GMCs create
small-scale density variations that are similar in amplitude to the dark-matter
induced variations but otherwise indistinguishable from it. These results
demonstrate that Pal 5 is a poor system for constraining the dark matter
substructure fraction and that observing streams further out in the halo will
be necessary to confidently detect dark matter subhalos.
