  We present Keck-MOSFIRE H and K spectra for a sample of 24 candidate
quiescent galaxies (QGs) at 3<z<4, identified from UVJ colors and photometric
redshifts in the ZFOURGE and 3DHST surveys. We obtain spectroscopic redshifts
for half of the sample, using absorption or emission lines, and confirm the
high accuracy of the photometric redshifts with a median error of 1.2%. Two
galaxies turn out to be dusty objects at lower redshifts (z<2.5), and are the
only two detected in the sub-mm with ALMA. High equivalent-width [OIII] was
observed in two galaxies, contributing up to 30% of the K-band flux and
mimicking the colors of an old stellar population. This implies a failure rate
of only 20% for the UVJ selection at these redshifts. Balmer absorption was
identified in 4 of the brighest galaxies, confirming the absence of OB stars.
Modeling all QGs with a wide range of star-formation histories, we find sSFR a
factor of 10 below the main sequence (MS) for all but one galaxy, and less than
0.01 Gyr$^{-1}$ for half of the sample. This is consistent with the H$\beta$
and [OII] luminosities, and the ALMA non-detections. We then find that these
QGs have quenched on average 300 Myr before observation, between z=3.5 and 5,
and that they formed at z~5.5 with a mean SFR~300 Msun/yr. Considering an
alternative selection of QGs based solely on the sSFR from SED modeling, we
find that galaxies a factor 10 below the MS are 40% more numerous than
UVJ-quiescent galaxies, implying that the UVJ selection is pure but incomplete.
Current models fail at reproducing our observations and underestimate either
the number density of QGs by more than an order of magnitude or the duration of
their quiescence by a factor two. Overall, these results confirm the existence
of an unexpected population of QGs at z>3, and offer the first insights on
their formation history. [abridged]
