Speech production under high cognitive load: A pilot study in combining prosody dynamics and pupillometric data
(2014) 6th Conference on Tone and Intonation in Europe — Location: Utrecht, the Netherlands (10.September.2014)
Files
No attached file found for this publication.
Details
- Authors
Christodoulides, George 
- Abstract
- In this pilot study we attempt to correlate the dynamics of prosodic features and prosodic events with an estimated index of cognitive load, obtained using pupillometry, during the task of simultaneous interpreting. Our objective is to study the particular characteristics of speech production under high cognitive load, both at the global and local level. Cognitive load (CL) is a multidimensional phenomenon defined as the amount of mental demand imposed by a particular task on the performer (Paas, 2003: 63), or as the perceived effort invested by her during the execution of the task. CL is the result of a task placing demands on cognitive systems with limited capacity, such as working memory (Baddeley 2007). Since cognitive overload negatively affects performance and may induce performance errors, the ability to estimate CL in real-time can be very useful, especially in high-stress environments. Pupillometry (Just et al. 2003) is a non-intrusive psycho-physiological method to quantify cognitive load, based on the observation that cognitive activity is correlated with pupil dilation (cf. Chen & Epps 2012). It has been used to study CL induced by language perception (e.g. Engonopoulos et al. 2013; Kun et al. 2013; Demberg et al. 2013), and especially during simultaneous interpreting (Seeber 2013). Simultaneous interpreting (SI) is a taxing cognitive task (Gile 1997), during which the interpreter is perceiving and producing speech under high CL. Previous studies on SI have revealed a particular prosodic profile, including: a large number of “low-rise non-final pitch movements” (Shlesinger 1994: 231); “long pauses [and a] high proportion of final pitch movements that indicate a continuation” (Ahrens 2005: 72); less numerous and longer silent pauses, more variable articulation rate and narrower pitch range, compared to the source speech (Christodoulides 2013). These findings are compatible with previous research comparing speech production under normal and high CL. It has been shown that CL affects pause duration and distribution, articulation rate, disfluencies and phonetic features (Berthold & Jameson 1999; Müller et al 2001; Jameson et al. 2009). Studies have proposed the use of segmental and prosodic features as classifiers, in order to distinguish speech produced under high CL, using machine learning techniques (Yin et al. 2008; Tet Fei Yap 2012). For the purposes of this study we recorded two professional conference interpreters who interpreted speeches from German and English into French (their mother tongue), and we simultaneously collected pupillometric data using the Pupil (Kassner & Patera 2012) portable eye tracker. This time-synchronised multi-track recording is transcribed and aligned to the phone level, allowing us to extract a series of segmental and prosodic features using a cascade of semi-automatic tools. The features studied include: temporal characteristics such as the distribution of silent pauses and speech rate; the distribution and prevalence of disfluencies; prominent syllables, including their patterning and density; pitch range and intonation patterns; prosodic phrasing and the relationship between major prosodic and syntactic boundaries. The results will be discussed based on the hypothesis that an increase in cognitive load, as measured by the eye-tracking data, will result in distinct patterns and combination of prosodic events in the interpreters’ speech. The objective data is complemented with a subjective evaluation of the task difficulty by the interpreters themselves (who are asked to identify parts of the original speech that they found difficult to interpret).
- Affiliations
Citations
Christodoulides, G. (2014). Speech production under high cognitive load: A pilot study in combining prosody dynamics and pupillometric data. 6th Conference on Tone and Intonation in Europe, Utrecht, the Netherlands. https://hdl.handle.net/2078.5/182686