  Mindfulness-based therapies have been shown to be effective in improving
mental health, and technology-based methods have the potential to expand the
accessibility of these therapies. To enable real-time personalized content
generation for mindfulness practice in these methods, high-quality
computer-synthesized text-to-speech (TTS) voices are needed to provide verbal
guidance and respond to user performance and preferences. However, the
user-perceived quality of state-of-the-art TTS voices has not yet been
evaluated for administering mindfulness meditation, which requires emotional
expressiveness. In addition, work has not yet been done to study the effect of
physical embodiment and personalization on the user-perceived quality of TTS
voices for mindfulness. To that end, we designed a two-phase human subject
study. In Phase 1, an online Mechanical Turk between-subject study (N=471)
evaluated 3 (feminine, masculine, child-like) state-of-the-art TTS voices with
2 (feminine, masculine) human therapists' voices in 3 different physical
embodiment settings (no agent, conversational agent, socially assistive robot)
with remote participants. Building on findings from Phase 1, in Phase 2, an
in-person within-subject study (N=94), we used a novel framework we developed
for personalizing TTS voices based on user preferences, and evaluated
user-perceived quality compared to best-rated non-personalized voices from
Phase 1. We found that the best-rated human voice was perceived better than all
TTS voices; the emotional expressiveness and naturalness of TTS voices were
poorly rated, while users were satisfied with the clarity of TTS voices.
Surprisingly, by allowing users to fine-tune TTS voice features, the
user-personalized TTS voices could perform almost as well as human voices,
suggesting user personalization could be a simple and very effective tool to
improve user-perceived quality of TTS voice.
