  Participatory design is a popular design technique that involves the end
users in the early stages of the design process to obtain user-friendly
gestural interfaces. Guessability studies followed by agreement analyses are
often used to elicit and comprehend the preferences (or gestures/proposals) of
the participants. Previous approaches to assess agreement, grouped the gestures
into equivalence classes and ignored the integral properties that are shared
between them. In this work, we represent the gestures using binary description
vectors to allow them to be partially similar. In this context, we introduce a
new metric referred to as soft agreement rate (SAR) to quantify the level of
consensus between the participants. In addition, we performed computational
experiments to study the behavior of our partial agreement formula and
mathematically show that existing agreement metrics are a special case of our
approach. Our methodology was evaluated through a gesture elicitation study
conducted with a group of neurosurgeons. Nevertheless, our formulation can be
applied to any other user-elicitation study. Results show that the level of
agreement obtained by SAR metric is 2.64 times higher than the existing
metrics. In addition to the mostly agreed gesture, SAR formulation also
provides the mostly agreed descriptors which can potentially help the designers
to come up with a final gesture set.
