  We provide a systematic measurement of the rest-frame UV continuum slope beta
over a wide range in redshift (z~2-6) and rest-frame UV luminosity (0.1-2L*) to
improve estimates of the SFR density at high redshift. We utilize the deep
optical and infrared data (ACS/NICMOS) over the CDF-S and HDF-N GOODS fields,
as well as the UDF for our primary UBVi "dropout" sample. We correct the
observed distributions for selection biases and photometric scatter. We find
that the UV-continuum slope of the most luminous galaxies is substantially
redder at z~2-4 than it is at z~5-6. Lower luminosity galaxies are also found
to be bluer than higher luminosity galaxies at z~2.5 and z~4. We do not find a
large number of galaxies with beta's as red as -1 in our dropout selections at
z~4, and particularly at z>~5, even though such sources could be readily
selected from our data. This suggests that star-forming galaxies at z>~5 almost
universally have very blue UV-continuum slopes, and that there are not likely
to be a substantial number of dust-obscured galaxies at z>~5 that are missed in
"dropout" searches. Using the same relation between UV-continuum slope and dust
extinction as found to be appropriate at z~0 and z~2, we estimate the average
dust extinction of galaxies as a function of redshift and UV luminosity in a
consistent way. We find that the estimated dust extinction increases
substantially with cosmic time for the most UV luminous galaxies, but remains
small (<~2x) at all times for lower luminosity galaxies. Because these same
lower luminosity galaxies dominate the luminosity density in the UV, the
overall dust extinction correction remains modest at all redshifts. We include
the contribution from ULIRGs in our SFR density estimates at z~2-6, but find
that they contribute only ~20% of the total at z~2.5 and <~10% at z>~4.
