  We describe an extension of the halo-based galaxy group-finding algorithm. We
add freedom to the algorithm in order to more accurately determine which
galaxies are central and which are satellites, and to provide unbiased
estimates of halo masses. We focus on determination of the galaxy-halo
relations for star-forming and quiescent galaxies. The added freedom in the
group-finding algorithm is self-calibrated using observations of
color-dependent galaxy clustering, as well as measurements of the total
satellite luminosity in deep imaging data around stacked samples of
spectroscopic central galaxies, L_sat. We test this approach on a series of
mocks that vary the galaxy-halo connection, including one mock constructed from
UniverseMachine results. Our self-calibrated algorithm shows marked improvement
over previous methods in estimating the color-dependent satellite fraction of
galaxies. It reduces the error in log M_halo for central galaxies by over a
factor of two, to <~0.2 dex. Through the L_sat data, it can quantify
differences in the luminosity-to-halo mass relations for star-forming and
quiescent galaxies, even for groups with only one spectroscopic member. Whereas
previous algorithms cannot constrain the scatter in L_gal at fixed M_halo, the
self-calibration technique can provide a robust lower limit to this scatter.
