  The substructures of the Galactic dark matter halo such as dark matter
subhalos and dwarf galaxies have very low velocity dispersions, which makes
them useful in constraining the scenario of Sommerfeld-enhanced dark matter
annihilation. We calculate the velocity distribution of dark matter particles
in dark matter halo substructures using the Eddington's formula with NFW
density profile. We parametrize the effect of Sommerfeld enhancement of s-wave
dark matter annihilation on the gamma-ray flux as the Sommerfeld-enhanced
J-factors, and explicitly calculate their values for 15 known dwarf spheroidal
galaxies. Using the Fermi-LAT 3FGL data on the unassociated point-sources and
the N-body simulation results on the dark matter subhalo distribution, we
derive upper limits on the dark matter annihilation cross sections with
Sommerfeld enhacement. Similar upper limits are derived from the Ferm-LAT
gamma-ray data on the dwarf spheroidal galaxies. We find that in a wide region
of parameter space, the constraints can be a few orders of magnitude more
stringent than that in the case without the Sommerfeld enhancement. For dark
matter annihilation dominantly into $e^+e^-$, $\mu^+\mu^-$, $\tau^{+}\tau^{-}$
and $b\bar b$, those constraints can exclude thermal relic dark matter for the
dark matter mass below about 1 TeV.
