Linguistic Corpus Annotation for Automatic Text Simplification Evaluation

Cardon, Rémi;Bibal, Adrien;Souza Wilkens, Rodrigo;Alfter, David;François, Thomas;et.al.
(2022) EMNLP 2022 — Location: Abu Dhabi, United Arab Emirates (7.December.2022)

Files

2022emnlp-main121.pdf
  • Open Access
  • Adobe PDF
  • 1.21 MB
Download

Details

Authors
  • Cardon, Rémiorcid-logoUCLouvain
    Author
  • Bibal, AdrienUCLouvain
    Author
  • Souza Wilkens, Rodrigoorcid-logoUCLouvain
    Author
  • Alfter, DavidUCLouvain
    Author
  • Norré, MagaliUCLouvain
    Author
  • Müller, AdelineUCLouvain
    Author
  • Watrin, PatrickUCLouvain
    Author
  • François, Thomasorcid-logoUCLouvain
    Author
Show more
Abstract
Evaluating automatic text simplification (ATS) systems is a difficult task that is either performed by automatic metrics or user-based evaluations. However, from a linguistic point-of-view, it is not always clear on what bases these evaluations operate. In this paper, we propose annotations of the ASSET corpus that can be used to shed more light on ATS evaluation. In addition to contributing with this resource, we show how it can be used to analyze SARI’s behavior and to re-evaluate existing ATS systems. We present our insights as a step to improve ATS evaluation protocols in the future.
Affiliations

Citations

  • APA
  • Chicago
  • FWB

Cardon, R., Bibal, A., Souza Wilkens, R., Alfter, D., Norré, M., Müller, A., Watrin, P., & François, T. (2022). Linguistic Corpus Annotation for Automatic Text Simplification Evaluation. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, p. 1842–1866. https://hdl.handle.net/2078.5/229842