Positive Semi-definite Embedding for Dimensionality Reduction and Out-of-Sample Extensions
(2022) SIAM Journal on Mathematics of Data Science — Vol. 4, n° 1, p. 153-178 (2022)
Files
4_Positive_Semi-definite_Embedding_for_Dimensionality_Reduction.pdf
postprint
Open Access - Adobe PDF
- 4.4 MB
Details
- Authors
- Abstract
- In machine learning or statistics, it is often desirable to reduce the dimensionality of a sample of data points in a high dimensional space Rd. This paper introduces a dimensionality reduction method where the embedding coordinates are the eigenvectors of a positive semi-definite kernel obtained as the solution of an infinite dimensional analogue of a semi-definite program. This embedding is adaptive and non-linear. We discuss this problem both with weak and strong smoothness assumptions about the learned kernel. A main feature of our approach is the existence of an out-of-sample extension formula of the embedding coordinates in both cases. This extrapolation formula yields an extension of the kernel matrix to a data-dependent Mercer kernel function. Our empirical results indicate that this embedding method is more robust with respect to the influence of outliers compared with a spectral embedding method.
- Affiliations
Citations
Fanuel, M., Aspeel, A., Delvenne, J.-C., & Suykens, J. A. K. (2022). Positive Semi-definite Embedding for Dimensionality Reduction and Out-of-Sample Extensions. SIAM Journal on Mathematics of Data Science, 4(1), 153-178. https://doi.org/10.1137/20M1370653 (Original work published 2022)