  This paper aims for a potential architectural improvement for multilingual
learning and asks: Can different tasks from different languages be modeled in a
monolithic framework, i.e. without any task/language-specific module? The
benefit of achieving this could open new doors for future multilingual
research, including allowing systems trained on low resources to be further
assisted by other languages as well as other tasks. We approach this goal by
developing a learning framework named Polyglot Prompting to exploit prompting
methods for learning a unified semantic space for different languages and tasks
with multilingual prompt engineering. We performed a comprehensive evaluation
of 6 tasks, namely topic classification, sentiment classification, named entity
recognition, question answering, natural language inference, and summarization,
covering 24 datasets and 49 languages. The experimental results demonstrated
the efficacy of multilingual multitask prompt-based learning and led to
inspiring observations. We also present an interpretable multilingual
evaluation methodology and show how the proposed framework, multilingual
multitask prompt training, works. We release all datasets prompted in the best
setting and code.
