A hybrid rule/model-based finite-state framework for normalizing SMS messages

Beaufort, Richard;Roekhaut, Sophie;Cougnon, Louise-Amélie;Fairon, Cédrick
(2010) The 48th Annual Meeting of the Association for Computational Linguistics (ACL 2010) — Location: Uppsala, Sweden, Uppsala University (11.July.2010)

Files

P10-1079-1.pdf
  • Open Access
  • Adobe PDF
  • 265.3 KB
Download

Details

Authors
Abstract
In recent years, research in natural language processing has increasingly focused on normalizing SMS messages. Different well-defined approaches have been proposed, but the problem remains far from being solved: best systems achieve a 11% Word Error Rate. This paper presents a method that shares similarities with both spell checking and machine translation approaches. The normalization part of the system is entirely based on models trained from a corpus. Evaluated in French by 10-fold-cross validation, the system achieves a 9.3% Word Error Rate and a 0.83 BLEU score.
Affiliations

Citations

  • APA
  • Chicago
  • FWB

Beaufort, R., Roekhaut, S., Cougnon, L.-A., & Fairon, C. (2010). A hybrid rule/model-based finite-state framework for normalizing SMS messages. In Jan Hajic, Sandra Carberry, Stephen Clark (ed.), Proceedings of The 48th Annual Meeting of the Association for Computational Linguistics (ACL 2010) (pp. 770-779). https://hdl.handle.net/2078.5/225207