  Much of the recent work developing formal methods techniques to specify or
learn the behavior of autonomous systems is predicated on a belief that formal
specifications are interpretable and useful for humans when checking systems.
Though frequently asserted, this assumption is rarely tested. We performed a
human experiment (N = 62) with a mix of people who were and were not familiar
with formal methods beforehand, asking them to validate whether a set of signal
temporal logic (STL) constraints would keep an agent out of harm and allow it
to complete a task in a gridworld capture-the-flag setting. Validation accuracy
was $45\% \pm 20\%$ (mean $\pm$ standard deviation). The ground-truth validity
of a specification, subjects' familiarity with formal methods, and subjects'
level of education were found to be significant factors in determining
validation correctness. Participants exhibited an affirmation bias, causing
significantly increased accuracy on valid specifications, but significantly
decreased accuracy on invalid specifications. Additionally, participants,
particularly those familiar with formal methods, tended to be overconfident in
their answers, and be similarly confident regardless of actual correctness.
  Our data do not support the belief that formal specifications are inherently
human-interpretable to a meaningful degree for system validation. We recommend
ergonomic improvements to data presentation and validation training, which
should be tested before claims of interpretability make their way back into the
formal methods literature.
