Sound Symbolism in sighted and Blind. The role of orthography and vision in sound-shape.

Barilari, Marco;Collignon, Olivier;Bottini, Roberto
(2018) European​ ​Workshop​ ​on​ ​Cognitive​ ​Neuropsychology​ ​2018 — Location: Bressanone - Italy (21.January.2018)

Files

No attached file found for this publication.

Details

Authors
Abstract
Introduction​:​ ​​Previous​ ​studies​ ​have​ ​suggested​ ​that​ ​non-arbitrary​ ​and​ ​seemingly​ ​universal​ ​sound-shape correspondences​ ​(sound​ ​symbolism;​ ​e.g.​ ​the​ ​bouba-kiki​ ​effect)​ ​are​ ​based​ ​on​ ​crossmodal​ ​mappings​ ​between​ ​the sound​ ​and​ ​the​ ​motor​ ​program​ ​needed​ ​to​ ​articulate​ ​that​ ​sound​ ​(Ramachandran​ ​et​ ​al.,​ ​2001).​ ​Yet,​ ​previous​ ​studies showed​ ​different​ ​patterns​ ​from​ ​the​ ​predictions​ ​of​ ​a​ ​pure​ ​articulatory​ ​account,​ ​calling​ ​for​ ​the​ ​effect​ ​of​ ​other factors.​ ​We​ ​suggest​ ​that​ ​deviation​ ​from​ ​articulation-based​ ​predictions​ ​may​ ​be​ ​due​ ​to​ ​the​ ​effect​ ​of​ ​letter​ ​shape. When​ ​listening​ ​to​ ​words​ ​people​ ​spontaneously​ ​activate​ ​their​ ​orthographic​ ​representation​ ​and​ ​the​ ​shape​ ​of​ ​written letters​ ​influences​ ​cross-modal​ ​mappings​ ​between​ ​shape​ ​and​ ​sound.​ ​We​ ​called​ ​this​ ​the​ ​‘Blending​ ​Orthography​ ​and Articulation​ ​Hypothesis’​ ​(BOAH). Methods​:​ ​​We​ ​tested​ ​39​ ​early​ ​blind​ ​(EB)​ ​and​ ​39​ ​sighted​ ​controls​ ​(SC).​ ​EB​ ​are​ ​exposed​ ​to​ ​an​ ​orthography (Braille)​ ​in​ ​which​ ​letters​ ​do​ ​not​ ​have​ ​spiky​ ​or​ ​round​ ​outlines.​ ​Exp.​ ​1​ ​assessed​ ​whether​ ​vision​ ​is​ ​necessary​ ​to develop​ ​sound​ ​symbolic​ ​association.​ ​Blind​ ​and​ ​blindfolded​ ​sighted​ ​participants​ ​(30​ ​per​ ​group)​ ​were​ ​asked​ ​to associate​ ​the​ ​words​ ​‘maluma’​ ​and​ ​‘takete’​ ​with​ ​various​ ​3D​ ​shapes. Exp.​ ​2​ ​was​ ​a​ ​direct​ ​test​ ​of​ ​BOAH.​ ​Participants​ ​(18​ ​per​ ​group)​ ​rated​ ​240​ ​verbal​ ​sounds​ ​as​ ​‘round’​ ​or​ ​‘pointy’. The​ ​sounds​ ​were​ ​composed​ ​by​ ​different​ ​consonant​ ​classes​ ​that​ ​are​ ​associated​ ​with​ ​various​ ​degrees​ ​of​ ​articulatory and​ ​orthographic​ ​spikiness.​ ​Mixed​ ​Effect​ ​Models​ ​were​ ​used​ ​to​ ​assess​ ​the​ ​relative​ ​weight​ ​of​ ​articulatory​ ​and orthographic​ ​spikiness​ ​in​ ​predicting​ ​sound-shape​ ​associations​ ​in​ ​both​ ​sighted​ ​and​ ​blind. Results​:​ ​​In​ ​Experiment​ ​1​ ​both​ ​groups​ ​showed​ ​a​ ​sound​ ​symbolic​ ​effect​ ​mapping​ ​shapes​ ​to​ ​words​ ​in​ ​the​ ​expected manner​ ​demonstrating​ ​that​ ​early​ ​loss​ ​of​ ​vision​ ​does​ ​not​ ​prevent​ ​the​ ​development​ ​of​ ​sound​ ​symbolism.​ ​Our​ ​results contrast​ ​with​ ​previous​ ​data​ ​which​ ​showed​ ​a​ ​significantly​ ​reduced​ ​effect​ ​in​ ​a​ ​heterogeneous​ ​population​ ​of​ ​visual impaired​ ​people​ ​(Fryer​ ​et.​ ​al,​ ​2014).​ ​In​ ​Experiment​ ​2,​ ​as​ ​predicted​ ​by​ ​BOAH,​ ​sound-shape​ ​associations​ ​were better​ ​explained​ ​in​ ​the​ ​blind​ ​by​ ​a​ ​pure​ ​articulatory​ ​model​ ​and​ ​in​ ​the​ ​sighted​ ​by​ ​a​ ​model​ ​that​ ​integrates articulatory​ ​and​ ​orthographic​ ​factors. Discussion​:​We​ ​suggest​ ​that​ ​previously​ ​reported​ ​pattern​ ​of​ ​shape-phoneme​ ​association​ ​can​ ​be​ ​better​ ​explained by​ ​an​ ​account​ ​that​ ​blends​ ​articulatory​ ​and​ ​orthographic​ ​information​ ​(BOAH).​ ​Phonological​ ​processing​ ​of​ ​words activates​ ​word’s​ ​orthographic​ ​representations​ ​that​ ​can​ ​influence​ ​the​ ​mapping​ ​between​ ​phonology​ ​and​ ​shape. Our​ ​results​ ​also​ ​demonstrate​ ​that​ ​early​ ​loss​ ​of​ ​vision​ ​does​ ​not​ ​prevent​ ​the​ ​development​ ​of​ ​sound​ ​symbolism,​ ​on the​ ​contrary​ ​early​ ​blinds​ ​may​ ​be​ ​highly​ ​sensitive​ ​to​ ​some​ ​iconic​ ​features​ ​of​ ​language.​ ​Indeed,​ ​sound​ ​symbolism may​ ​help​ ​blind​ ​children​ ​to​ ​solve​ ​referential​ ​ambiguity​ ​problems​ ​during​ ​language​ ​learning​ ​that​ ​are​ ​largely​ ​based on​ ​visual​ ​statistics​ ​in​ ​sighted​ ​(Smith​ ​et​ ​al.,​ ​2010). References​:​ ​​Ramachandran​ ​VS​ ​et​ ​al.​ ​J​ ​Consc​ ​Studies.​ ​(2001)​ ​8(12),​ ​3 Fryer​ ​L​ ​et​ ​al.​ ​Cogn​ ​(2014)​ ​132(2),​ ​164 Smith​ ​LB​ ​et​ ​al.​ ​Cog​ ​science​ ​(2010)​ ​34(7),​ ​1287
Affiliations

Citations

  • APA
  • Chicago
  • FWB

Barilari, M., Collignon, O., & Bottini, R. (2018). Sound Symbolism in sighted and Blind. The role of orthography and vision in sound-shape. European​ ​Workshop​ ​on​ ​Cognitive​ ​Neuropsychology​ ​2018, Bressanone - Italy. https://hdl.handle.net/2078.5/100467