  Since about 2003, captchas have been widely used as a barrier against bots,
while simultaneously annoying great multitudes of users worldwide. As their use
grew, techniques to defeat or bypass captchas kept improving, while captchas
themselves evolved in terms of sophistication and diversity, becoming
increasingly difficult to solve for both bots and humans. Given this
long-standing and still-ongoing arms race, it is important to investigate
usability, solving performance, and user perceptions of modern captchas. In
this work, we do so via a large-scale (over 3, 600 distinct users) 13-month
real-world user study and post-study survey. The study, conducted at a large
public university, was based on a live account creation and password recovery
service with currently prevalent captcha type: reCAPTCHAv2.
  Results show that, with more attempts, users improve in solving checkbox
challenges. For website developers and user study designers, results indicate
that the website context directly influences (with statistically significant
differences) solving time between password recovery and account creation. We
consider the impact of participants' major and education level, showing that
certain majors exhibit better performance, while, in general, education level
has a direct impact on solving time. Unsurprisingly, we discover that
participants find image challenges to be annoying, while checkbox challenges
are perceived as easy. We also show that, rated via System Usability Scale
(SUS), image tasks are viewed as "OK", while checkbox tasks are viewed as
"good". We explore the cost and security of reCAPTCHAv2 and conclude that it
has an immense cost and no security. Overall, we believe that this study's
results prompt a natural conclusion: reCAPTCHAv2 and similar reCAPTCHA
technology should be deprecated.
