  How can we enable users to create effective, perception-driven task plans for
collaborative robots? We conducted a 35-person user study with the Behavior
Tree-based CoSTAR system to determine which strategies for end user creation of
generalizable robot task plans are most usable and effective. CoSTAR allows
domain experts to author complex, perceptually grounded task plans for
collaborative robots. As a part of CoSTAR's wide range of capabilities, it
allows users to specify SmartMoves: abstract goals such as "pick up component A
from the right side of the table." Users were asked to perform pick-and-place
assembly tasks with either SmartMoves or one of three simpler baseline versions
of CoSTAR. Overall, participants found CoSTAR to be highly usable, with an
average System Usability Scale score of 73.4 out of 100. SmartMove also helped
users perform tasks faster and more effectively; all SmartMove users completed
the first two tasks, while not all users completed the tasks using the other
strategies. SmartMove users showed better performance for incorporating
perception across all three tasks.
