Can MDS rival with t-SNE by using the symmetric Kullback-Leibler divergence across neighborhoods as a pseudo-distance?

Lee, John;Lambert, Pierre;Couplet, Edouard;Merveille, Pierre;Verleysen, Michel;et.al.
(2025) 33th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning — Location: Bruges, Belgium (23.April.2025)

Files

Lee2025_MDS_symmetric_KL.pdf
  • Open Access
  • Adobe PDF
  • 7.09 MB
Download

Details

Authors
Show more
Abstract
Local methods of dimensionality reduction like neighborhood embedding (NE) and t-SNE in particular outperform older global approaches such as stress-based multi-dimensional scaling (MDS). Stochastic neighborhoods are less sensitive than distances to statistical variations between spaces with strongly different dimensionalities, making a match across them very difficult. Here, we take inspiration from those stochastic neighborhoods in order to devise a pseudo-distance that is less prone to concentration than the Euclidean distance. For two points in the high-dimensional data space, it is defined as the symmetrized Kullback-Leibler divergence across the (stochastic) neighborhoods of the two points (SKLAN). Plugging the SKLAN in a method of stress-based MDS, we compare quantitatively t-SNE, MDS with all Euclidean distances, and MDS with SKLAN & Euclidean distances on several data sets. The results show that SKLAN allows MDS to perform competitively with t-SNE.
Affiliations

Citations

  • APA
  • Chicago
  • FWB

Lee, J., Lambert, P., Couplet, E., Merveille, P., Journaux, L., Mulders, D., de Bodt, C., & Verleysen, M. (2025). Can MDS rival with t-SNE by using the symmetric Kullback-Leibler divergence across neighborhoods as a pseudo-distance? ESANN 2025 proceedings. Published. 33th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium. https://doi.org/10.14428/esann/2025.ES2025-174