The BEA 2024 Shared Task on the Multilingual Lexical Simplification Pipeline

Shardlow, Matthew;Alva-Manchego, Fernando;Batista-Navarro, Riza;Bott, Stefan;Saggion, Horacio;et.al.
(2024) Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024) — Location: Mexico City, Mexico (20.June.2024)

Files

2024bea-151.pdf
  • Open Access
  • Adobe PDF
  • 218.25 KB
Download

Details

Authors
  • Shardlow, MatthewManchester Metropolitan University
    Author
  • Alva-Manchego, FernandoCardiff University
    Author
  • Batista-Navarro, RizaUniversity of Manchester
    Author
  • Bott, StefanUniversitat Pompeu Fabra
    Author
  • Cardon, Rémiorcid-logoUCLouvain
    Author
  • François, Thomasorcid-logoUCLouvain
    Author
  • Saggion, HoracioUniversitat Pompeu Fabra
    Author
Show more
Abstract
We report the findings of the 2024 Multilingual Lexical Simplification Pipeline shared task. We released a new dataset comprising 5,927 instances of lexical complexity prediction and lexical simplification on common contexts across 10 languages, split into trial (300) and test (5,627). 10 teams participated across 2 tracks and 10 languages with 233 runs evaluated across all systems. Five teams participated in all languages for the lexical complexity prediction task and 4 teams participated in all languages for the lexical simplification task. Teams employed a range of strategies, making use of open and closed source large language models for lexical simplification, as well as feature-based approaches for lexical complexity prediction. The highest scoring team on the combined multilingual data was able to obtain a Pearson’s correlation of 0.6241 and an ACC@1@Top1 of 0.3772, both demonstrating that there is still room for improvement on two difficult sub-tasks of the lexical simplification pipeline.
Affiliations

Citations

  • APA
  • Chicago
  • FWB

Shardlow, M., Alva-Manchego, F., Batista-Navarro, R., Bott, S., Calderon Ramirez, S., Cardon, R., François, T., Hayakawa, A., Horbach, A., Hülsing, A., Ide, Y., Marvin Imperial, J., Nohejl, A., North, K., Occhipinti, L., Peréz Rojas, N., Raihan, N., Ranasinghe, T., Solis Salazar, M., et al. (2024). The BEA 2024 Shared Task on the Multilingual Lexical Simplification Pipeline. In Ekaterina Kochmar, Marie Bexte, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante, Anaïs Tack, Victoria Yaneva, Zheng Yua (ed.), Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024) (Association for Computational Linguistics, p. p. 571-589). Association for Computational Linguistics. https://hdl.handle.net/2078.5/231586