  In this paper, a novel switching pushing skill algorithm is proposed to
improve the efficiency of planar non-prehensile manipulation, which draws
inspiration from human pushing actions and comprises two sub-problems, i.e.,
discrete decision-making of pushing point and continuous feedback control of
pushing action. In order to solve the sub-problems above, a combination of
Model Predictive Control (MPC) and Deep Reinforcement Learning (DRL) method is
employed. Firstly, the selection of pushing point is modeled as a Markov
decision process,and an off-policy DRL method is used by reshaping the reward
function to train the decision-making model for selecting pushing point from a
pre-constructed set based on the current state. Secondly, a motion constraint
region (MCR) is constructed for the specific pushing point based on the
distance from the target, followed by utilizing the MPC controller to regulate
the motion of the object within the MCR towards the target pose. The trigger
condition for switching the pushing point occurs when the object reaches the
boundary of the MCR under the pushing action. Subsequently, the pushing point
and the controller are updated iteratively until the target pose is reached. We
conducted pushing experiments on four distinct object shapes in both simulated
and physical environments to evaluate our method. The results indicate that our
method achieves a significantly higher training efficiency, with a training
time that is only about 20% of the baseline method while maintaining around the
same success rate. Moreover, our method outperforms the baseline method in
terms of both training and execution efficiency of pushing operations, allowing
for rapid learning of robot pushing skills.
