  Previous soft tissue manipulation studies assumed that the grasping point was
known and the target deformation can be achieved. During the operation, the
constraints are supposed to be constant, and there is no obstacles around the
soft tissue. To go beyond these assumptions, a deep reinforcement learning
framework with prior knowledge is proposed for soft tissue manipulation under
unknown constraints, such as the force applied by fascia. The prior knowledge
is represented through an intuitive manipulation strategy. As an action of the
agent, a regulator factor is used to coordinate the intuitive approach and the
deliberate network. A reward function is designed to balance the exploration
and exploitation for large deformation. Successful simulation results verify
that the proposed framework can manipulate the soft tissue while avoiding
obstacles and adding new position constraints. Compared with the soft
actor-critic (SAC) algorithm, the proposed framework can accelerate the
training procedure and improve the generalization.
