Detection of Large Language Model Contamination with Tabular Data
(2025) Symposium on Intelligent Data Analysis 2025 (IDA 2025) — Location: Konstanz, Germany (7.May.2025)
Files
DetectionofLLMContaminationwithTabularData-IDA2025.pdf
Open Access - Adobe PDF
- 412.75 KB
Details
- Authors
- Abstract
- Large Language Models (LLMs) are increasingly used to solve machine learning tasks on tabular data, such as classification tasks. Using standard benchmarks, recent studies have shown impressive performance for such tasks. However, in this paper we study a major concern with the use of standard benchmarks: LLMs may have also been trained using these benchmarks, that is, these LLMs are contaminated. While previous work mostly focused on the GPT models, we propose a methodology to evaluate whether a large range of LLMs is contaminated. We propose new tests to detect contamination with tabular data over two aspects: knowledge and memorization. We also design an algorithm to parse the answers of the LLMs and detect hints of contamination. Our experiments conclude that the bigger and more closed-source an LLM is, the more likely it is contaminated.
- Affiliations
Citations
Ronval, B., Dupont, P., & Nijssen, S. (2025). Detection of Large Language Model Contamination with Tabular Data. Advances in Intelligent Data Analysis XXIII 23nd International Symposium on Intelligent Data Analysis, IDA 2025, Konstanz, Germa, 234-245. https://doi.org/10.1007/978-3-031-91398-3_18 (Original work published 2025)