Accounting for the non-normality of asset returns remains challenging in robust portfolio optimization. In this article, we tackle this problem by assessing the risk of the portfolio through the ''amount of randomness'' conveyed by its returns. We achieve this by using an objective function that relies on the exponential of Rényi entropy, an information-theoretic criterion that precisely quantifies the uncertainty embedded in a distribution, accounting for all moments. Compared to Shannon entropy, Rényi entropy features a parameter that can be tuned to play around the notion of uncertainty. It is shown to control the relative contributions of the central and tail parts of the distribution in the measure. We further rely on a non-parametric estimator of the exponential Rényi entropy that extends a robust sample-spacings estimator initially designed for Shannon entropy. A portfolio selection application illustrates that minimizing Rényi entropy yields portfolios that outperform standard minimum risk portfolios in terms of Sharpe ratio and higher-moment preferences.