  Learning performant robot manipulation policies can be challenging due to
high-dimensional continuous actions and complex physics-based dynamics. This
can be alleviated through intelligent choice of action space. Operational Space
Control (OSC) has been used as an effective task-space controller for
manipulation. Nonetheless, its strength depends on the underlying modeling
fidelity, and is prone to failure when there are modeling errors. In this work,
we propose OSC for Adaptation and Robustness (OSCAR), a data-driven variant of
OSC that compensates for modeling errors by inferring relevant dynamics
parameters from online trajectories. OSCAR decomposes dynamics learning into
task-agnostic and task-specific phases, decoupling the dynamics dependencies of
the robot and the extrinsics due to its environment. This structure enables
robust zero-shot performance under out-of-distribution and rapid adaptation to
significant domain shifts through additional finetuning. We evaluate our method
on a variety of simulated manipulation problems, and find substantial
improvements over an array of controller baselines. For more results and
information, please visit https://cremebrule.github.io/oscar-web/.
