  We present deep Hubble Space Telescope (HST) photometry of the ultra-faint
dwarf (UFD) galaxies Pegasus III (Peg III) and Pisces II (Psc II), two of the
most distant satellites in the halo of the Milky Way (MW). We measure the
structure of both galaxies, derive mass-to-light ratios with newly determined
absolute magnitudes, and compare our findings to expectations from UFD-mass
simulations. For Peg III, we find an elliptical half-light radius of
$a_h=1.88^{+0.42}_{-0.33}$ arcminutes ($118^{+31}_{-30}$ pc) and
$M_V{=}{-4.17}^{+0.19}_{-0.22}$; for Psc II, we measure
$a_h{=}1.31^{+0.10}_{-0.09}$ arcminutes ($69\pm8$ pc) and
$M_V{=}{-4.28}^{+0.19}_{-0.16}$. We do not find any morphological features that
indicate a significant interaction between the two has occurred, despite their
close separation of only $\sim$40 kpc. Using proper motions (PMs) from Gaia
early Data Release 3, we investigate the possibility of any past association by
integrating orbits for the two UFDs in a MW-only and a combined MW and Large
Magellanic Cloud (LMC) potential. We find that including the gravitational
influence of the LMC is crucial, even for these outer-halo satellites, and that
a possible orbital history exists where Peg III and Psc II experienced a close
($\sim$10-20 kpc) passage about each other just over $\sim$1 Gyr ago, followed
by a collective passage around the LMC ($\sim$30-60 kpc) just under $\sim$1 Gyr
ago. Considering the large uncertainties on the PMs and the restrictive priors
imposed to derive them, improved PM measurements for Peg III and Psc II will be
necessary to clarify their relationship. This would add to the rare findings of
confirmed pairs of satellites within the Local Group.
