Crawling by Readability Level
(2016) Proceedings of the 12th International Conference on Computational Processing of the Portuguese Language — ISBN: [9783319415512], 306-318, published
Files
2016 - PROPOR 2016 - Crawling by Readability Level.pdf
Open Access - Adobe PDF
- 418.56 KB
Details
- Authors
- Abstract
- The availability of annotated corpora for research in the area of Readability Assessment is still very limited. On the other hand, the Web is increasingly being used by researchers as a source of written content to build very large and rich corpora, in the Web as Corpus (WaC) initiative. This paper proposes a framework for automatic generation of large corpora classified by readability. It adopts a supervised learning method to incorporate a readability filter based in features with low computational cost to a crawler, to collect texts targeted at a specific reading level. We evaluate this framework by comparing a readability-assessed web crawled corpus to a reference corpus (Both corpora are available in http://www. inf.ufrgs.br/pln/resource/CrawlingByReadabilityLevel.zip.). The results obtained indicate that these features are good at separating texts from level 1 (initial grades) from other levels. As a result of this work two Portuguese corpora were constructed: the Wikilivros Readability Corpus, classified by grade level, and a crawled WaC classified by readability level.
- Affiliations
Citations
Filho, J., Wilkens, R., Zilio, L., Idiart, M., & Villavicencio, A. (2016). Crawling by Readability Level. In João Silva, Ricardo Ribeiro, Paulo Quaresma, André Adami, António Branco (ed.), Proceedings of the 12th International Conference on Computational Processing of the Portuguese Language (pp. 306-318). Springer International Publishing. https://doi.org/10.1007/978-3-319-41552-9_31