  We evaluate the performance of the SDSS DR8 redMaPPer photometric cluster
catalog by comparing it to overlapping X-ray and SZ selected catalogs from the
literature. We confirm the redMaPPer photometric redshifts are nearly unbiased
(<\Delta z> < 0.005), have low scatter (\sigma_z ~ 0.006-0.02, depending on
redshift), and have a low catastrophic failure rate (~ 1%). Both the
T_X-\lambda\ and Mgas-\lambda\ scaling relations are consistent with a mass
scatter of \sigma_{\ln M|\lambda} ~ 25%, albeit with a ~ 1% outlier rate due to
projection effects. This failure rate is somewhat lower than that expected for
the full cluster sample, but is consistent with the additional selection
effects introduced by our reliance on X-ray and SZ selected reference cluster
samples. Where the redMaPPer DR8 catalog is volume limited (z < 0.35), the
catalog is 100% complete above T_X > 3.5 keV, and L_X > 2\times 10^{44} erg/s,
decreasing to 90% completeness at L_X ~ 10^{43} erg/s. All rich (\lambda >
100), low redshift (z < 0.25) redMaPPer clusters are X-ray detected in the
ROSAT All Sky Survey (RASS), and 86% of the clusters are correctly centered.
Compared to other SDSS photometric cluster catalogs, redMaPPer has the highest
completeness and purity, and the best photometric redshift performance, though
some algorithms do achieve comparable performance to redMaPPer in subsets of
the above categories and/or in limited redshift ranges. The redMaPPer richness
is clearly the one that best correlates with X-ray temperature and gas mass.
Most algorithms (including redMaPPer) have very similar centering performance,
with only one exception which performs worse.
