  We present an analysis of the dust attenuation of star forming galaxies at
$z=2.5-4.0$ through the relationship between the UV spectral slope ($\beta$),
stellar mass ($M_{\ast}$) and the infrared excess
(IRX$=L_{\rm{IR}}/L_{\rm{UV}}$) based on far-infrared continuum observations
from the Atacama Large Millimeter/sub-millimeter Array (ALMA). Our study
exploits the full ALMA archive over the COSMOS field processed by the
A$^3$COSMOS team, which includes an unprecedented sample of $\sim1500$ galaxies
at $z\sim3$ as primary or secondary targets in ALMA band 6 or 7 observations
with a median continuum sensitivity of 126 $\rm{\mu Jy/beam}$ (1$\sigma$). The
detection rate is highly mass dependent, decreasing drastically below $\log
(M_{\ast}/M_{\odot})=10.5$. The detected galaxies show that the IRX-$\beta$
relationship of massive ($\log M_{\ast}/M_{\odot} > 10$) main sequence galaxies
at $z=2.5-4.0$ is consistent with that of local galaxies, while starbursts are
generally offset by $\sim0.5\,{\rm dex}$ to larger IRX values. At the low mass
end, we derive upper limits on the infrared luminosities through stacking of
the ALMA data. The combined IRX-$M_{\ast}$ relation at
$\rm{log\,(M_{\ast}/M_{\odot})>9}$ exhibits a significantly steeper slope than
reported in previous studies at similar redshifts, implying little dust
obscuration at $\log M_{\ast}/M_{\odot}<10$. However, our results are
consistent with early measurements at $z\sim5.5$, indicating a potential
redshift evolution between $z\sim2$ and $z\sim6$. Deeper observations targeting
low mass galaxies will be required to confirm this finding.
