Nesterov momentum and gradient normalization to improve t-SNE convergence and neighborhood preservation, without early exaggeration

Lambert, Pierre;Lee, John;Couplet, Edouard;De Bodt, Cyril
(2023) 31st European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning — Location: Bruges, Belgium (4.October.2023)

Files

Lambert2023_ES2023-147.pdf
  • Open Access
  • Adobe PDF
  • 1.7 MB
Download

Details

Authors
Abstract
Student t-distributed stochastic neighbor embedding (t-SNE) finds low-dimensional data representations allowing visual exploration of data sets. t-SNE minimises a cost function with a custom two-phase gradient descent. The first phase is called early exaggeration and involves a hyper-parameter whose value can be tricky and time-consuming to set. This paper proposes another way to optimise the cost function without early exaggeration. Empirical evaluation shows that the proposed method of optimization converges faster and yields competitive results in terms of neighborhood preservation.
Affiliations

Citations

  • APA
  • Chicago
  • FWB

Lambert, P., Lee, J., Couplet, E., & De Bodt, C. (2023). Nesterov momentum and gradient normalization to improve t-SNE convergence and neighborhood preservation, without early exaggeration. ESANN proceedings, 1(1), 1-6. https://doi.org/10.14428/esann/2023.ES2023-147 (Original work published 2023)