  A core challenge for an autonomous agent acting in the real world is to adapt
its repertoire of skills to cope with its noisy perception and dynamics. To
scale learning of skills to long-horizon tasks, robots should be able to learn
and later refine their skills in a structured manner through trajectories
rather than making instantaneous decisions individually at each time step. To
this end, we propose the Soft Actor-Critic Gaussian Mixture Model (SAC-GMM), a
novel hybrid approach that learns robot skills through a dynamical system and
adapts the learned skills in their own trajectory distribution space through
interactions with the environment. Our approach combines classical robotics
techniques of learning from demonstration with the deep reinforcement learning
framework and exploits their complementary nature. We show that our method
utilizes sensors solely available during the execution of preliminarily learned
skills to extract relevant features that lead to faster skill refinement.
Extensive evaluations in both simulation and real-world environments
demonstrate the effectiveness of our method in refining robot skills by
leveraging physical interactions, high-dimensional sensory data, and sparse
task completion rewards. Videos, code, and pre-trained models are available at
http://sac-gmm.cs.uni-freiburg.de.
