Sign language representation in the visual brain
(2026) Bilingualism Matters - Bimodal Session — Location: Milan (28.April.2026)
Files
Abstract_BilingualismMatter2026.docx
Open Access - Microsoft Word XML
- 18.57 MB
Details
- Authors
- Abstract
- (en) Sign languages are richly visual linguistic systems that rely on multiple articulators—the hands, body, and face—to convey meaning. Unlike spoken languages, sign languages are perceived directly through vision and require the brain to extract linguistic information from complex, dynamic visual signals(1). These visual articulators engage the brain’s visual infero-temporal cortex(2), which is organized into category-selective areas specialized for faces, bodies, and words. While the involvement of classical fronto-temporal language regions in sign language processing is well established, the specific contribution of Ventral Occipito Temporal Cortex (VOTC) to linguistic versus visual aspects of sign processing remains underexplored. The VOTC encompasses several functionally specialized regions(3), including the fusiform face area (FFA) and occipital face area (OFA); the extrastriate body area (EBA) and hand-selective occipital regions; and the visual word form area (VWFA). In sign language, manual and bodily movements, together with facial expressions, convey linguistic content at multiple levels of structure. However, VOTC activation during sign perception has often been attributed to domain-general visual processing, rather than language-specific mechanisms(2,5). Whether activity in these regions reflects simple perceptual encoding of articulator movements or deeper linguistic processing remains an open question central to understanding how visual and linguistic representations interact in the brain. In other words, whether and where expertise with sign language changes the visual brain responses to signs (versus pseudo-signs) remains unknown. In this study, we investigated the involvement of individually-localized category-selective VOTC regions—including the anterior and posterior VWFA, FFA, OFA, EBA, and a hand-selective occipital region as well as the early visual cortex (EVC)—during the perception of real non-iconic signs versus pseudo-signs (figure 1). Given that recent research suggests that the anterior, but not posterior, portion of the VWFA may specifically support lexical and semantic processing, VWFA was split into anterior and posterior sub-ROIs(4). Using representational similarity analysis (RSA) applied to fMRI data from hearing signers (Children of Deaf Adults - CODAs) and hearing non-signers, we characterized fine-grained neural representations associated with sign perception. Importantly, the RSA examined sign versus pseudo-sign distinctions after controlling for low-level visual and kinematic factors. This approach allowed us to isolate neural patterns reflecting linguistic rather than purely perceptual properties of the stimuli. Our results revealed stronger distinction in the brain representation of real versus pseudo-signs in signers than in non-signers in the anterior VWFA, FFA, OFA, EBA, and the hand-selective occipital region (but not in pVWFA and V1), indicating that sign language expertise sharpens representational specificity within visual–linguistic networks (figure 2). The anterior VWFA (but not the posterior VWFA), showed a pattern consistent with sensitivity to the linguistic properties of signs rather than their purely visual form, suggesting a modality-independent role in processing symbolic visual language. Complementary searchlight analyses uncovered a distributed network spanning ventral occipito-temporal and parietal cortices that differentiated real from pseudo-signs, specifically in signers compared to non-signers. Overall, our findings reveal that expertise with sign language changes the brain response to signs beyond the classical « language network », instead engaging a distributed occipito-temporal–parietal network that integrates perceptual and linguistic processes. More precisely, our results unequivocally demonstrate that category-selective VOTC regions, traditionally viewed as domain-specific visual processors, are instead flexibly recruited to support the complex visual–linguistic demands of sign language.
- Affiliations
Citations
Mattioni, S., Alice Van Audenhaege, Hans Op de Beeck, & Collignon, O. (2026). Sign language representation in the visual brain (NA, ed.). https://hdl.handle.net/2078.5/277325