  The non-linear gravitational wave (GW) memory effect is a distinct prediction
in general relativity. While the effect has been well studied for comparable
mass binaries, it has mostly been overlooked for intermediate mass ratio
inspirals (IMRIs). We offer a comprehensive analysis of the phenomenology and
detectability of memory effects, including contributions from subdominant
harmonic modes, in heavy IMRIs consisting of a stellar mass black hole and an
intermediate mass black hole. When formed through hierarchical mergers, for
example when a GW190521-like remnant captures a stellar mass black hole, IMRI
systems have a large total mass, large spin on the primary, and possibly
residual eccentricity; features that potentially raise the prospect for memory
detection. We compute both the displacement and spin non-linear GW memory from
the $m \neq 0$ gravitational waveforms computed within a black hole
perturbation theory framework that is partially calibrated to numerical
relativity waveforms. We probe the dependence of memory effects on mass ratio,
spin, and eccentricity and consider the detectability of a memory signal from
IMRIs using current and future GW detectors. We find that (i) while
eccentricity introduces additional features in both displacement and spin
memory, it does not appreciatively change the prospects of detectability, (ii)
including higher modes into the memory computation can increase singal-to-noise
(SNR) values by about 7\% in some cases, (iii) the SNR from displacement memory
dramatically increases as the spin approaches large, positive values, (iv) spin
memory from heavy IMRIs would, however, be difficult to detect with future
generation detectors even from highly spinning systems. Our results suggest
that hierarchical binary black hole mergers may be a promising source for
detecting memory and could favorably impact memory forecasts.
