CreoleVal: Multilingual Multitask Benchmarks for Creoles
Name
tacl_a_00682.pdf
Description
Published version
Size
515.29 KB
Format
Adobe PDF
Checksum (MD5)
acf1711714945f25e626389967b3f620
Author(s)
Lent, Heather
Tatariya, Kushal
Dabre, Raj
Chen, Yiyi
Fekete, Marcell
Ploeger, Esther
Zhou, Li
Armstrong, Ruth-Ann
Eijansantos, Abee
Malau, Catriona
Date Issued
September 4, 2024
Journal
Transactions of the Association for Computational Linguistics
Publisher
MIT Press
Citation
Heather Lent, Kushal Tatariya, Raj Dabre, Yiyi Chen, Marcell Fekete, Esther Ploeger, Li Zhou, Ruth-Ann Armstrong, Abee Eijansantos, Catriona Malau, Hans Erik Heje, Ernests Lavrinovics, Diptesh Kanojia, Paul Belony, Marcel Bollmann, Loïc Grobol, Miryam de Lhoneux, Daniel Hershcovich, Michel DeGraff, Anders Søgaard, Johannes Bjerva; CreoleVal: Multilingual Multitask Benchmarks for Creoles. Transactions of the Association for Computational Linguistics 2024; 12 950–978.
Version
Final published version
Abstract
Creoles represent an under-explored and marginalized group of languages, with few available resources for NLP research. While the genealogical ties between Creoles and a number of highly resourced languages imply a significant potential for transfer learning, this potential is hampered due to this lack of annotated data. In this work we present CreoleVal, a collection of benchmark datasets spanning 8 different NLP tasks, covering up to 28 Creole languages; it is an aggregate of novel development datasets for reading comprehension relation classification, and machine translation for Creoles, in addition to a practical gateway to a handful of preexisting benchmarks. For each benchmark, we conduct baseline experiments in a zero-shot setting in order to further ascertain the capabilities and limitations of transfer learning for Creoles. Ultimately, we see CreoleVal as an opportunity to empower research on Creoles in NLP and computational linguistics, and in general, a step towards more equitable language technology around the globe.
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1162/tacl_a_00682
