  Robotics policies are always subjected to complex, second order dynamics that
entangle their actions with resulting states. In reinforcement learning (RL)
contexts, policies have the burden of deciphering these complicated
interactions over massive amounts of experience and complex reward functions to
learn how to accomplish tasks. Moreover, policies typically issue actions
directly to controllers like Operational Space Control (OSC) or joint PD
control, which induces straightline motion towards these action targets in task
or joint space. However, straightline motion in these spaces for the most part
do not capture the rich, nonlinear behavior our robots need to exhibit,
shifting the burden of discovering these behaviors more completely to the
agent. Unlike these simpler controllers, geometric fabrics capture a much
richer and desirable set of behaviors via artificial, second order dynamics
grounded in nonlinear geometry. These artificial dynamics shift the
uncontrolled dynamics of a robot via an appropriate control law to form
behavioral dynamics. Behavioral dynamics unlock a new action space and safe,
guiding behavior over which RL policies are trained. Behavioral dynamics enable
bang-bang-like RL policy actions that are still safe for real robots, simplify
reward engineering, and help sequence real-world, high-performance policies. We
describe the framework more generally and create a specific instantiation for
the problem of dexterous, in-hand reorientation of a cube by a highly actuated
robot hand.
