Observing very-short-term speaking fluency development in computer-delivered interviews
(2024) The 33rd Conference of the European Second Language Association (EuroSLA 2024) — Location: Montpellier, France (3.July.2024)
Files
No attached file found for this publication.
Details
- Authors
- Abstract
- BACKGROUND. Utterance fluency, as a dimension of L2 speaking, has been demonstrated to be a key component of L2 proficiency and being able to measure it precisely is essential for testing and research. While various utterance fluency metrics have been proposed, compared and validated against various standards in the literature (e.g., Segalowitz, French & Guay, 2017; Tavakoli, Nakatsuhara & Hunter, 2020), their sensitivity to short-term learning gains has been more rarely studied and, when it has, it was usually during interventions lasting for 4 weeks or more (e.g., Campbell, Colin and McCormack, 2016; Segalowitz et al, 2017). The present study presents a methodology for measuring and analysing speech fluency in computer-delivered interviews that allowed us to observe significant L2 fluency development over less than 3 weeks and only 2 hours of intervention. METHODS. We used a custom-made computer-delivered oral interview consisting of 26 short questions to record individually and simultaneously 215 teenage low-intermediate learners of French in a pre- and a posttest, separated by 1 to 3 weeks. In the experimental group, learners practised the L2 through written interactions with automated agents (chatbots) in a dialogue-based game. The resulting 12'000 audio files were transcribed by automatic speech recognition, manually corrected and annotated for a series of disfluencies. We computed both signal-based (via de Jong & Wempe 2009) and transcription-based fluency metrics, with many variations in terms of pruning (e.g., including or excluding filled pauses, false starts, repetitions, self-talk, L1 words...). We compared learning gains from pre- to posttest and across conditions for most utterance fluency metrics (speech rate, articulation rate, syllable duration, length of runs, duration of runs, duration of silent pauses, silent pausing rate, speech-time ratio, pause duration, etc.). To go beyond a single metric, we computed a composite fluency index based on the first component of a principal component analysis of these metrics. RESULTS. Results show that the most sensitive metric, speech rate, is well able to detect a significant gain in L2 speaking fluency between the pre- and posttest (d = 0.51, p < .001), i.e. after 1 to 3 weeks, but fails to observe a significant effect between the experimental and the control group, which shows that part of the pre-post difference is due to a task repetition effect (d = 0.39). However, the analysis made on the first principal component shows promising sensitivity and sufficient power to distinguish the task-retask effect from the intervention effect. DISCUSSION. Our semi-automated method for data collection and analysis of speaking fluency offers promising precision and sensitivity for measuring very-short-term L2 proficiency gains. A stronger fluency-oriented intervention would certainly produce even more effects in such a timeframe.
- Affiliations
Citations
Bibauw, S. (2024). Observing very-short-term speaking fluency development in computer-delivered interviews. The 33rd Conference of the European Second Language Association (EuroSLA 2024), Montpellier, France. https://hdl.handle.net/2078.5/233277