  We report a 4.8$\sigma$ measurement of the cross-correlation signal between
the cosmic microwave background (CMB) lensing convergence reconstructed from
measurements of the CMB polarization made by the POLARBEAR experiment and the
infrared-selected galaxies of the Herschel-ATLAS survey. This is the first
measurement of its kind. We infer a best-fit galaxy bias of $b = 5.76 \pm
1.25$, corresponding to a host halo mass of $\log_{10}(M_h/M_\odot)
=13.5^{+0.2}_{-0.3}$ at an effective redshift of $z \sim 2$ from the
cross-correlation power spectrum. Residual uncertainties in the redshift
distribution of the sub-mm galaxies are subdominant with respect to the
statistical precision. We perform a suite of systematic tests, finding that
instrumental and astrophysical contaminations are small compared to the
statistical error. This cross-correlation measurement only relies on CMB
polarization information that, differently from CMB temperature maps, is less
contaminated by galactic and extra-galactic foregrounds, providing a clearer
view of the projected matter distribution. This result demonstrates the
feasibility and robustness of this approach for future high-sensitivity CMB
polarization experiments.
