  A popular paradigm in robotic learning is to train a policy from scratch for
every new robot. This is not only inefficient but also often impractical for
complex robots. In this work, we consider the problem of transferring a policy
across two different robots with significantly different parameters such as
kinematics and morphology. Existing approaches that train a new policy by
matching the action or state transition distribution, including imitation
learning methods, fail due to optimal action and/or state distribution being
mismatched in different robots. In this paper, we propose a novel method named
$REvolveR$ of using continuous evolutionary models for robotic policy transfer
implemented in a physics simulator. We interpolate between the source robot and
the target robot by finding a continuous evolutionary change of robot
parameters. An expert policy on the source robot is transferred through
training on a sequence of intermediate robots that gradually evolve into the
target robot. Experiments on a physics simulator show that the proposed
continuous evolutionary model can effectively transfer the policy across robots
and achieve superior sample efficiency on new robots. The proposed method is
especially advantageous in sparse reward settings where exploration can be
significantly reduced. Code is released at https://github.com/xingyul/revolver.
