  BeppoSAX has revealed a population of faint neutron star X-ray transients in
the Galactic bulge. King conjectured that these neutron stars are accreting
from brown dwarfs (BD) with a time-averaged mass transfer rate <\dot M>\approx
10^{-11} M_sun/year. We show that the orbital parameters of the 401 Hz
accreting millisecond pulsar SAX J1808.4-3658 support this hypothesis. A
main-sequence mass donor requires a nearly face-on inclination and a higher
<\dot M> than observed. However, the range of allowed inclinations is
substantially relaxed and the predicted mass transfer rate is consistent with
that observed if a hot 0.05 M_sun BD is the donor. We then explain the BD
radius required (0.13 R_\sun) to fill the Roche lobe. All transiently accreting
neutron stars have a minimum luminosity in quiescence set by the time-averaged
mass transfer rate onto the neutron star. We show that the constant heating of
the BD by this emission can maintain the higher entropy implied by a 0.13
R_\sun radius. Our work bolsters the case that SAX J1808.4-3658 is a progenitor
to compact ms radio pulsar binaries (e.g. like those found in 47 Tuc). The very
low <\dot M>'s implies that the progenitors to these radio pulsars are
long-lived (Gyrs) transient systems, rather than short-lived (Myrs)
Eddington-limited accretors. Hence, the accreting progenitor population to
millisecond radio pulsars in 47 Tuc could still be present and found in
quiescence with Chandra.(abridged)
