  Many core collapse supernovae (SNe) with hydrogen-poor and low-mass ejecta,
such as ultra-stripped SNe and type Ibn SNe, are observed to interact with
dense circumstellar material (CSM). These events likely arise from the
core-collapse of helium stars which have been heavily stripped by a binary
companion and ejected significant mass during the last weeks to years of their
lives. In helium star models run to days before core-collapse, we identify a
range of helium core masses $\approx 2.5 -3 M_{\odot}$ whose envelopes expand
substantially due to helium shell burning while the core undergoes neon and
oxygen burning. When modeled in binary systems, the rapid expansion of these
helium stars induces extremely high rates of late-stage mass transfer ($\dot{M}
\gtrsim 10^{-2} M_\odot/{\rm yr}$) beginning weeks to decades before
core-collapse. We consider two scenarios for producing CSM in these systems:
either mass transfer remains stable and mass loss is driven from the system in
the vicinity of the accreting companion, or mass transfer becomes unstable and
causes a common envelope event (CEE) through which the helium envelope is
unbound. The ensuing CSM properties are consistent with the CSM masses ($\sim
10^{-2}-1 M_\odot$) and radii ($\sim 10^{13}-10^{16} {\rm cm}$) inferred for
ultra-stripped SNe and several type Ibn SNe. Furthermore, systems that undergo
a CEE could produce short-period NS binaries that merge in less than 100 Myr.
