  Deep Reinforcement Learning (DRL) has proven effective in learning control
policies using robotic grippers, but much less practical for solving the
problem of grasping with dexterous hands -- especially on real robotic
platforms -- due to the high dimensionality of the problem. In this work, we
focus on the multi-fingered grasping task with the anthropomorphic hand of the
iCub humanoid. We propose the RESidual learning with PREtrained CriTics
(RESPRECT) method that, starting from a policy pre-trained on a large set of
objects, can learn a residual policy to grasp a novel object in a fraction
($\sim 5 \times$ faster) of the timesteps required to train a policy from
scratch, without requiring any task demonstration. To our knowledge, this is
the first Residual Reinforcement Learning (RRL) approach that learns a residual
policy on top of another policy pre-trained with DRL. We exploit some
components of the pre-trained policy during residual learning that further
speed-up the training. We benchmark our results in the iCub simulated
environment, and we show that RESPRECT can be effectively used to learn a
multi-fingered grasping policy on the real iCub robot. The code to reproduce
the experiments is released together with the paper with an open source
license.
