  A crucial problem for partial sky analysis of CMB polarization is the $E$-$B$
leakage problem. Such leakage arises from the presence of `ambiguous' modes
that satisfy properties of both $E$ and $B$ modes. Solving this problem is
critical for primordial polarization $B$ mode detection in partial sky CMB
polarization experiments. In this work we introduce a new method for reducing
the leakage. We demonstrate that if we complement the $E$-mode information
outside the observation patch with ancillary data from full-sky CMB
observations, we can reduce and even effectively remove the $E$-to-$B$ leakage.
For this objective, we produce $E$-mode Stokes $QU$ maps from Wiener filtered
full-sky intensity and polarization CMB observations. We use these maps to fill
the sky region that is not observed by the ground-based experiment of interest,
and thus complement the partial sky Stokes $QU$ maps. Since the $E$-mode
information is now available on the full sky we see a significant reduction in
the $E$-to-$B$ leakage. We evaluate on simulated data sets the performance of
our method for a `shallow' $f_\text{sky}=8\%$, and a `deep' $f_\text{sky}=2\%$
northern hemisphere sky patch, with AliCPT-like properties, and a LSPE-like
$f_\text{sky}=30\%$ sky patch, by combining those observations with Planck-like
full sky polarization maps. We find that our method outperforms the standard
and the pure-$B$ method pseudo-$C_\ell$ estimators for all of our simulations.
Our new method gives unbiased estimates of the $B$-mode power spectrum
through-out the entire multipole range with near-optimal pseudo-$C_\ell$ errors
for $\ell>20$. We also study the application of our method to the CMB-S4
experiment combined with LiteBIRD-like full sky data, and show that using
signal-dominated full sky $E$-mode data we can eliminate the $E$-to-$B$ leakage
problem.
