Multilingual NLP¶
Multilingual natural language processing addresses the challenge of building NLP systems that work effectively across diverse languages. This spans approaches including multilingual pre-training (BERT, XLM-R), cross-lingual transfer learning, zero-shot and few-shot adaptation, and explicit multilingual evaluation protocols that surface disparities across language communities.
A central concern in multilingual NLP is ensuring systems perform fairly across languages, particularly for lower-resourced and typologically distant languages. Evaluation often reveals that "good" systems on high-resource languages (usually English) degrade substantially on others, and that performance gaps are not randomly distributed but correlate with language resource availability and linguistic distance from English.
Key papers¶
- Harm Is Behavioural, Moderation Is Lexical: How Language and Culture Decide Whom Automated Safety Systems Protect and Whom They Silence — Evaluates multilingual content moderation systems; demonstrates that performance gaps between best- and worst-served languages are systematic (52.7 percentage points for explicit hate, 45.7–57.1 for implicit hate), and proposes evaluation protocol requiring native-speaker annotation and per-language reporting to surface disparities.
Related topics¶
- Multilingual Models (technical approach using shared representations)
- Low-Resource Language Processing (focus on data scarcity and transfer)