  Autonomy in robotic surgery is very challenging in unstructured environments,
especially when interacting with deformable soft tissues. The main difficulty
is to generate model-based control methods that account for deformation
dynamics during tissue manipulation. Previous works in vision-based perception
can capture the geometric changes within the scene, however, model-based
controllers integrated with dynamic properties, a more accurate and safe
approach, has not been studied before. Considering the mechanic coupling
between the robot and the environment, it is crucial to develop a registered,
simulated dynamical model. In this work, we propose an online, continuous,
real-to-sim registration method to bridge 3D visual perception with
position-based dynamics (PBD) modeling of tissues. The PBD method is employed
to simulate soft tissue dynamics as well as rigid tool interactions for
model-based control. Meanwhile, a vision-based strategy is used to generate 3D
reconstructed point cloud surfaces based on real-world manipulation, so as to
register and update the simulation. To verify this real-to-sim approach, tissue
experiments have been conducted on the da Vinci Research Kit. Our real-to-sim
approach successfully reduces registration error online, which is especially
important for safety during autonomous control. Moreover, it achieves higher
accuracy in occluded areas than fusion-based reconstruction.
