  The prevalence of social media presents a growing opportunity to collect and
analyse examples of English varieties. Whilst usage of these varieties was -
and, in many cases, still is - used only in spoken contexts or hard-to-access
private messages, social media sites like Twitter provide a platform for users
to communicate informally in a scrapeable format. Notably, Indian English
(Hinglish), Singaporean English (Singlish), and African-American English (AAE)
can be commonly found online. These varieties pose a challenge to existing
natural language processing (NLP) tools as they often differ orthographically
and syntactically from standard English for which the majority of these tools
are built. NLP models trained on standard English texts produced biased
outcomes for users of underrepresented varieties. Some research has aimed to
overcome the inherent biases caused by unrepresentative data through techniques
like data augmentation or adjusting training models.
  We aim to address the issue of bias at its root - the data itself. We curate
a dataset of tweets from countries with high proportions of underserved English
variety speakers, and propose an annotation framework of six categorical
classifications along a pseudo-spectrum that measures the degree of standard
English and that thereby indirectly aims to surface the manifestations of
English varieties in these tweets. Following best annotation practices, our
growing corpus features 170,800 tweets taken from 7 countries, labeled by
annotators who are from those countries and can communicate in
regionally-dominant varieties of English. Our corpus highlights the accuracy
discrepancies in pre-trained language identifiers between western English and
non-western (i.e., less standard) English varieties. We hope to contribute to
the growing literature identifying and reducing the implicit demographic
discrepancies in NLP.
