  We introduce RoboNinja, a learning-based cutting system for multi-material
objects (i.e., soft objects with rigid cores such as avocados or mangos). In
contrast to prior works using open-loop cutting actions to cut through
single-material objects (e.g., slicing a cucumber), RoboNinja aims to remove
the soft part of an object while preserving the rigid core, thereby maximizing
the yield. To achieve this, our system closes the perception-action loop by
utilizing an interactive state estimator and an adaptive cutting policy. The
system first employs sparse collision information to iteratively estimate the
position and geometry of an object's core and then generates closed-loop
cutting actions based on the estimated state and a tolerance value. The
"adaptiveness" of the policy is achieved through the tolerance value, which
modulates the policy's conservativeness when encountering collisions,
maintaining an adaptive safety distance from the estimated core. Learning such
cutting skills directly on a real-world robot is challenging. Yet, existing
simulators are limited in simulating multi-material objects or computing the
energy consumption during the cutting process. To address this issue, we
develop a differentiable cutting simulator that supports multi-material
coupling and allows for the generation of optimized trajectories as
demonstrations for policy learning. Furthermore, by using a low-cost force
sensor to capture collision feedback, we were able to successfully deploy the
learned model in real-world scenarios, including objects with diverse core
geometries and soft materials.
