  A long-standing challenge for a robotic manipulation system operating in
real-world scenarios is adapting and generalizing its acquired motor skills to
unseen environments. We tackle this challenge employing hybrid skill models
that integrate imitation and reinforcement paradigms, to explore how the
learning and adaptation of a skill, along with its core grounding in the scene
through a learned keypoint, can facilitate such generalization. To that end, we
develop Keypoint Integrated Soft Actor-Critic Gaussian Mixture Models (KIS-GMM)
approach that learns to predict the reference of a dynamical system within the
scene as a 3D keypoint, leveraging visual observations obtained by the robot's
physical interactions during skill learning. Through conducting comprehensive
evaluations in both simulated and real-world environments, we show that our
method enables a robot to gain a significant zero-shot generalization to novel
environments and to refine skills in the target environments faster than
learning from scratch. Importantly, this is achieved without the need for new
ground truth data. Moreover, our method effectively copes with scene
displacements.
