Exploring the neural basis of phonological representations from sounds and vision
(2023) IMRF (International Multisensory Research Forum) — Location: Brussels (27.June.2023)
Files
posterIMRF23.pdf
Closed Access - Adobe PDF
- 2.44 MB
Details
- Authors
Van Audenhaege, Alice 
Cerpelloni, Filippo Gau, Remi
- Abstract
- Speech is a multisensory signal that we can decipher from the voice and/or the lips. If the successive computational steps necessary to transform the auditory signal into meaningful language representations have been extensively explored, little is known on how the visual input of speech is processed in the brain; and how auditory and visual speech information are combined to converge onto a unified linguistic percept. In this study, we aim to identify brain regions that are involved in auditory (phonemes) and visual (visemes) phonology and explore whether some brain regions can be considered as multisensory abstract phonological regions supporting both auditory and visual phonological representations. We rely on functional magnetic resonance imaging (fMRI) in healthy adults to classify brain activity patterns evoked by phonemes and visemes. Preliminary results suggest that a network of visual, motor, auditory and frontal regions are involved in viseme recognition. Interestingly, auditorily defined phonological regions (in superior temporal gyrus - STG) seem to be involved in visual phonological representations as well. Moreover, overlap between auditory and visual decoding in mid- and posterior STG and in motor cortex indicate that these regions could be involved in the integration of auditory and visual speech phonology.
- Affiliations
Citations
Van Audenhaege, A., & Collignon, O. (2023). Exploring the neural basis of phonological representations from sounds and vision. IMRF (International Multisensory Research Forum), Brussels. https://hdl.handle.net/2078.5/213864