Gradient representations in ReLU networks as similarity functions
(2021) ESANN 2021 - European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning — Location: Online event (Bruges, Belgium) (6.October.2021)
Files
ES2021-153.pdf
Open Access - Adobe PDF
- 1.63 MB
Details
- Authors
- Abstract
- Feed-forward networks can be interpreted as mappings with linear decision surfaces at the level of the last layer. We investigate how the tangent space of the network can be exploited to refine the decision in case of ReLU (Rectangular Linear Unit) activations. We show that a simple Riemannian metric parametrized on the parameters of the network forms a similarity function at least as good as the original network and we suggest a sparse metric to increase the similarity gap.
- Affiliations
Citations
Daroczy, B. Z., & Rácz, D. (2021). Gradient representations in ReLU networks as similarity functions. ESANN 2021 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Published. ESANN 2021 - European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Online event (Bruges, Belgium). https://doi.org/10.14428/esann/2021.es2021-153