  Conducting experiments with diverse participants in their native languages
can uncover insights into culture, cognition, and language that may not be
revealed otherwise. However, conducting these experiments online makes it
difficult to validate self-reported language proficiency. Furthermore, existing
proficiency tests are small and cover only a few languages. We present an
automated pipeline to generate vocabulary tests using text from Wikipedia. Our
pipeline samples rare nouns and creates pseudowords with the same low-level
statistics. Six behavioral experiments (N=236) in six countries and eight
languages show that (a) our test can distinguish between native speakers of
closely related languages, (b) the test is reliable ($r=0.82$), and (c)
performance strongly correlates with existing tests (LexTale) and self-reports.
We further show that test accuracy is negatively correlated with the linguistic
distance between the tested and the native language. Our test, available in
eight languages, can easily be extended to other languages.
