Exploration of Potential New Benchmark for Fairness Evaluation in Europe
(2024) EWAF’24: European Workshop on Algorithmic Fairness — Location: Mainz, Germany (1.July.2024)
Files
paper_45_proceedings.pdf
Open Access - Adobe PDF
- 955.14 KB
Details
- Authors
- Abstract
- With the increase use of artificial intelligence systems and the associated concerns regarding automated discrimination, research in the field of fairness has increased in the past years. To evaluate their work in fair machine learning,researchers have often been using the same three datasets (Adult, COMPAS, and German credit) as benchmarks. However, those datasets each present serious limitations. In this work, we first explore what other datasets could potentially be used as replacement, specifically in a European context. We then use an experimental approach to compare Adult and COMPAS with a new candidate, Student Performance (a.k.a Student Alcohol Consumption). Our early results highlight the scarcity of easily accessible European datasets suitable as benchmarks for fairness evaluation of problems with positive or negative outcome, as well as the high influence dataset selection can have on experimental results.
- Affiliations
Citations
Legast, M., Koutsoviti Koumeri, L., Yousefi, Y., & Legay, A. (2024). Exploration of Potential New Benchmark for Fairness Evaluation in Europe. C E U R Workshop Proceedings, Vol-3908, 7. https://hdl.handle.net/2078.5/241652 (Original work published 2024)