The role of context and age in the use of French causal connectives by humans and LLMs
(2026) (2026)
Files
Abstract_SLE_2026_Updated.pdf
Open Access - Adobe PDF
- 290.58 KB
Details
- Authors
- Abstract
- Large Language Models (LLMs) have achieved remarkable success across a wide range of Natural Language Processing tasks. Yet their ability to handle fine-grained pragmatic distinctions, especially those relying on subtle contextual and subjective cues, remains an open question (Gao et al. 2021). This paper investigates this issue through a classic but unresolved problem in French linguistics: the choice between the causal connectives parce que and car ('because'). While traditionally distinguished along semantic and pragmatic lines-parce que being associated with objective causality and car with subjective reasoning (Degand & Pander Maat 2003, Pit 2003)-recent research challenges this dichotomy, suggesting greater overlap and ongoing change in usage, particularly among younger speakers (Nazarenko 2000, Zufferey et al. 2018, Blochowiak & Grisot, 2022). Building on recent pragmatic accounts that conceptualize subjectivity as an emergent property derived from multiple cues distributed across discourse (Blochowiak et al. 2020, 2024), we test the hypothesis that connective choice depends on contextual information extending beyond the sentence level. We address this hypothesis through two complementary studies: a Minimal Context Study and an Extended Context Study, comparing human judgments with LLM behavior. In the Minimal Context Study, we used as a test set a corpus of 420 French excerpts drawn from journalistic texts (Le Monde) and SMS messages (Blochowiak et al. 2020, SMS4science Fairon et al. 2006). In each excerpt, the original connective (parce que or car) was removed and systematically replaced with a <MASK> token. Three human participants and two LLMs were asked to predict the missing connective. Results from three experiments show that human annotators perform poorly on isolated sentences, with low accuracy and agreement, especially in SMS data. GPT-3.5, tested via few-shot prompting, exhibits inconsistent and overall weaker performance than humans, though it performs unexpectedly better on SMS than on newswire texts. As training corpus, we used a dataset of 10,000 sentences (5,000 containing car and 5,000 containing parce que) sourced from 4,200 SMS4science text messages and 5,800 Belgian news articles. A fine-tuned CamemBERT model trained on 10,000 examples from the training corpus outperforms both humans and GPT-3.5, achieving an accuracy of 66.7%, with a strong effect of text genre. The Extended Context Study, currently ongoing, investigates whether access to broader discourse context and speaker age modulate connective choice. Using the same corpus enriched with preceding and following sentences, we compare younger (20-30) and older (39-59) speakers across genres. The same task is then applied to LLMs, including GPT-4.5 and a fine-tuned CamemBERT model, to assess whether models benefit from extended context and whether their behavior aligns with specific age profiles. In sum, the results from the Minimal Context Study had shown that CamemBERT outperforms LLMs and humans in low context environments. The ongoing Extended Context Study, enriched context and speaker variation, should inform us whether LLMs align more closely with specific age profiles.
- Affiliations
Citations
Blochowiak, J., Zeng, R.-Y., de Marneffe, M.-C., & Degand, E. (2026). The role of context and age in the use of French causal connectives by humans and LLMs. Submitted. https://doi.org/10.21203/rs.3.rs-8078247/v1 (Original work published 2026)