Disentangling the Construct of Phraseological Sophistication: a Validation Study with Human Judgments Work-in-progress Report
(2026) LxGr2026 — Location: United Kingdom (2.July.2026)
Files
LxGr26_GabrielaVaughan.pdf
Open Access - Adobe PDF
- 507.86 KB
Details
- Authors
- Abstract
- Linguistic complexity is regarded as a major research variable in Applied Linguistics (AL), being used to describe L2 performance, assess L2 proficiency, and trace L2 development (Housen & Kuiken, 2009; Paquot, 2019). Existing research on complexity has largely focused on developing measures of lexical and syntactic complexity (Bulté & Housen, 2012), neglecting those at the lexis-grammar interface. At the same time, phraseology has widely been recognised as playing an important role in language proficiency and development (Wray, 2002). Learner Corpus Research has established that more proficient second language learners (L2) use a wider range of collocations and more sophisticated recurrent word combinations than less proficient learners (Bestgen & Granger, 2018; Paquot, 2019). Despite this, there remains a lack of studies which theorise and operationalise linguistic complexity at this interface in phrasal-level language production. To address this gap, Paquot (2019) proposed the construct of phraseological complexity, defined by both range (diversity) and sophistication. According to this conceptualisation, a learner’s text is considered more complex when it contains a higher proportion of sophisticated, varied (diversity) phraseological units rather than frequent repetitions of common word combinations (Paquot, 2019). Automated corpus-based measures of phraseological sophistication are particularly promising linguistic measures for distinguishing among proficiency levels (CEFR) (Paquot, 2019; Vandeweerd et al., 2021; Jiang et al., 2021). However, the construct validity of these measures has not been adequately addressed, as it remains unclear to what extent they align with human perceptions of phraseological sophistication (and complexity). From a validity-argument perspective, this constitutes a significant gap in learner corpus research, as a linguistic construct must reflect a perceptual reality to be valid, which can only be achieved by capturing human perception of the targeted construct (Jarvis, 2017). This work-in-progress study addresses this issue by gathering data on the human perception of phraseological sophistication in collocations. Phraseological sophistication is theorised to be measured across three dimensions: register specificity, association strength, and frequency (Paquot & Naets, 2025). A set of 120 verb–object collocations was extracted from a Wikicorpus and balanced across these dimensions using corpus-based measures. Thirty expert judges evaluated these collocations using a comparative judgment task (Thwaites & Paquot, 2024). Comparative judgment is a novel approach that has proven useful for evaluating complex linguistic constructs (Crossley et al., 2023; Zhang & Lu, 2024). This study aims (1) to determine the extent to which L1 English speakers share intuitions about phraseological sophistication, (2) to determine the influence of register specificity, association strength, and frequency on those intuitions, and (3) to establish the construct validity of this corpus-based measure. Drawing on L2 phraseological research, we expect that judges will rank collocations that are register-specific, strongly associated, and relatively uncommon as more sophisticated. Analysis will employ Scale Separation Reliability and infit statistics to determine whether judges share intuitions regarding phraseological sophistication (Pollitt, 2012). Regression modelling will be used to investigate which dimensions of phraseological sophistication most predict human judgments. The task will also require judges to provide comments regarding their decision-making process, enabling a qualitative analysis of what guided their decision-making. This Comparative Judgment task will be conducted in Spring 2026 using the No More Marking platform, and this presentation will report the first results from the study. By integrating human perception data with automatic corpus-based measures, this project aims to enhance our theoretical and methodological understanding of phraseological complexity and clarify its role in Second Language Acquisition research.
- Affiliations
Edge Hill University LxGr
Citations
Vaughan, G., & Paquot, M. (2026, July 3). Disentangling the Construct of Phraseological Sophistication: a Validation Study with Human Judgments Work-in-progress Report. LxGr2026, United Kingdom. https://hdl.handle.net/2078.5/279495