  The ability of robots to manipulate objects relies heavily on their aptitude
for visual perception. In domains characterized by cluttered scenes and high
object variability, most methods call for vast labeled datasets, laboriously
hand-annotated, with the aim of training capable models. Once deployed, the
challenge of generalizing to unfamiliar objects implies that the model must
evolve alongside its domain. To address this, we propose a novel framework that
combines Semi-Supervised Learning (SSL) with Learning Through Interaction
(LTI), allowing a model to learn by observing scene alterations and leverage
visual consistency despite temporal gaps without requiring curated data of
interaction sequences. As a result, our approach exploits partially annotated
data through self-supervision and incorporates temporal context using
pseudo-sequences generated from unlabeled still images. We validate our method
on two common benchmarks, ARMBench mix-object-tote and OCID, where it achieves
state-of-the-art performance. Notably, on ARMBench, we attain an
$\text{AP}_{50}$ of $86.37$, almost a $20\%$ improvement over existing work,
and obtain remarkable results in scenarios with extremely low annotation,
achieving an $\text{AP}_{50}$ score of $84.89$ with just $1 \%$ of annotated
data compared to $72$ presented in ARMBench on the fully annotated counterpart.
