  Phishing websites are everywhere, and countermeasures based on static
blocklists cannot cope with such a threat. To address this problem,
state-of-the-art solutions entail the application of machine learning (ML) to
detect phishing websites by checking if they visually resemble webpages of
well-known brands. These techniques have achieved promising results in research
and, consequently, some security companies began to deploy them also in their
phishing detection systems (PDS). However, ML methods are not perfect and some
samples are bound to bypass even production-grade PDS.
  In this paper, we scrutinize whether 'genuine phishing websites' that evade
'commercial ML-based PDS' represent a problem "in reality". Although nobody
likes landing on a phishing webpage, a false negative may not lead to serious
consequences if the users (i.e., the actual target of phishing) can recognize
that "something is phishy". Practically, we carry out the first user-study
(N=126) wherein we assess whether unsuspecting users (having diverse
backgrounds) are deceived by 'adversarial' phishing webpages that evaded a real
PDS. We found that some well-crafted adversarial webpages can trick most
participants (even IT experts), albeit others are easily recognized by most
users. Our study is relevant for practitioners, since it allows prioritizing
phishing webpages that simultaneously fool (i) machines and (ii) humans --
i.e., their intended targets.
