  Recent trends in NLP research have raised an interest in linguistic
code-switching (CS); modern approaches have been proposed to solve a wide range
of NLP tasks on multiple language pairs. Unfortunately, these proposed methods
are hardly generalizable to different code-switched languages. In addition, it
is unclear whether a model architecture is applicable for a different task
while still being compatible with the code-switching setting. This is mainly
because of the lack of a centralized benchmark and the sparse corpora that
researchers employ based on their specific needs and interests. To facilitate
research in this direction, we propose a centralized benchmark for Linguistic
Code-switching Evaluation (LinCE) that combines ten corpora covering four
different code-switched language pairs (i.e., Spanish-English, Nepali-English,
Hindi-English, and Modern Standard Arabic-Egyptian Arabic) and four tasks
(i.e., language identification, named entity recognition, part-of-speech
tagging, and sentiment analysis). As part of the benchmark centralization
effort, we provide an online platform at ritual.uh.edu/lince, where researchers
can submit their results while comparing with others in real-time. In addition,
we provide the scores of different popular models, including LSTM, ELMo, and
multilingual BERT so that the NLP community can compare against
state-of-the-art systems. LinCE is a continuous effort, and we will expand it
with more low-resource languages and tasks.
