  A stochastic gravitational wave background (SGWB) will affect the CMB
anisotropies via weak lensing. Unlike weak lensing due to large scale structure
which only deflects photon trajectories, a SGWB has an additional effect of
rotating the polarization vector along the trajectory. We study the relative
importance of these two effects, deflection \& rotation, specifically in the
context of E-mode to B-mode power transfer caused by weak lensing due to SGWB.
Using weak lensing distortion of the CMB as a probe, we derive constraints on
the spectral energy density ($\Omega_{GW}$) of the SGWB, sourced at different
redshifts, without assuming any particular model for its origin. We present
these bounds on $\Omega_{GW}$ for different power-law models characterizing the
SGWB, indicating the threshold above which observable imprints of SGWB must be
present in CMB.
