  As robots are deployed in complex situations, engineers and end users must
develop a holistic understanding of their behaviors, capabilities, and
limitations. Some behaviors are directly optimized by the objective function.
They often include success rate, completion time or energy consumption. Other
behaviors -- e.g., collision avoidance, trajectory smoothness or motion
legibility -- are typically emergent but equally important for safe and
trustworthy deployment. Designing an objective which optimizes every aspect of
robot behavior is hard. In this paper, we advocate for systematic analysis of a
wide array of behaviors for holistic understanding of robot controllers and, to
this end, propose a framework, RoCUS, which uses Bayesian posterior sampling to
find situations where the robot controller exhibits user-specified behaviors,
such as highly jerky motions. We use RoCUS to analyze three controller classes
(deep learning models, rapidly exploring random trees and dynamical system
formulations) on two domains (2D navigation and a 7 degree-of-freedom arm
reaching), and uncover insights to further our understanding of these
controllers and ultimately improve their designs.
