Please, don’t forget the difference and the confidence interval when seeking for the state-of-the-art status
(2022) 13th Conference on Language Resources and Evaluation (LREC 2022)
Files
2022lrec-1640copie.pdf
Open Access - Adobe PDF
- 895.04 KB
Details
- Authors
- Abstract
- This paper argues for the widest possible use of bootstrap confidence intervals for comparing NLP system performances instead of the state-of-the-art status (SOTA) and statistical significance testing. Their main benefits are to draw attention to the difference in performance between two systems and to help assessing the degree of superiority of one system over another. Two cases studies, one comparing several systems and the other based on a K-fold cross-validation procedure, illustrate these benefits.
- Affiliations
Citations
Bestgen, Y. (2022). Please, don’t forget the difference and the confidence interval when seeking for the state-of-the-art status. Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), p. 956-962. https://hdl.handle.net/2078.5/231011