CENTAL at TSAR-2022 Shared Task: How Does Context Impact BERT-Generated Substitutions for Lexical Simplification?

Souza Wilkens, Rodrigo;Alfter, David;Cardon, Rémi;Gribomont, Isabelle;François, Thomas;et.al.
(2022) Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022) — Location: Abu Dhabi, United Arab Emirates (6.December.2022)

Files

2022tsar-125.pdf
  • Open Access
  • Adobe PDF
  • 1.73 MB
Download

Details

Authors
Show more
Abstract
Lexical simplification is the task of substituting a difficult word with a simpler equivalent for a target audience. This is currently commonly done by modeling lexical complexity on a continuous scale to identify simpler alternatives to difficult words. In the TSAR shared task, the organizers call for systems capable of generating substitutions in a zero-shot-task context, for English, Spanish and Portuguese. In this paper, we present the solution we (the CENTAL team) proposed for the task. We explore the ability of BERT-like models to generate substitution words by masking the difficult word. To do so, we investigate various context enhancement strategies, that we combined into an ensemble method. We also explore different substitution ranking methods. We report on a post-submission analysis of the results and present our insights for potential improvements. The code for all our experiments is available at https://gitlab.com/Cental-FR/cental-tsar2022.
Affiliations

Citations

  • APA
  • Chicago
  • FWB

Souza Wilkens, R., Alfter, D., Cardon, R., Gribomont, I., Bibal, A., Watrin, P., de Marneffe, M.-C., & François, T. (2022). CENTAL at TSAR-2022 Shared Task: How Does Context Impact BERT-Generated Substitutions for Lexical Simplification? In Sanja Štajner, Horacio Saggion, Daniel Ferrés, Matthew Shardlow, Kim Cheng Sheang, Kai North, Marcos Zampieri, Wei Xu (ed.), Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022) (Association for Computational Linguistics, p. p. 231 - 238). Association for Computational Linguistics. https://hdl.handle.net/2078.5/229835