  Twitter has become a vital social media platform while an ample amount of
malicious Twitter bots exist and induce undesirable social effects. Successful
Twitter bot detection proposals are generally supervised, which rely heavily on
large-scale datasets. However, existing benchmarks generally suffer from low
levels of user diversity, limited user information and data scarcity.
Therefore, these datasets are not sufficient to train and stably benchmark bot
detection measures. To alleviate these problems, we present TwiBot-20, a
massive Twitter bot detection benchmark, which contains 229,573 users,
33,488,192 tweets, 8,723,736 user property items and 455,958 follow
relationships. TwiBot-20 covers diversified bots and genuine users to better
represent the real-world Twittersphere. TwiBot-20 also includes three modals of
user information to support both binary classification of single users and
community-aware approaches. To the best of our knowledge, TwiBot-20 is the
largest Twitter bot detection benchmark to date. We reproduce competitive bot
detection methods and conduct a thorough evaluation on TwiBot-20 and two other
public datasets. Experiment results demonstrate that existing bot detection
measures fail to match their previously claimed performance on TwiBot-20, which
suggests that Twitter bot detection remains a challenging task and requires
further research efforts.
