Input variable selection is a key preprocess step in any I/O modelling problem. Normally, better generalization performance is obtained when unneeded parameters coming from irrelevant or redundant variables are eliminated. Information theory provides a robust theoretical framework for performing input variable selection thanks to the concept of mutual information. Nevertheless, for continuous variables, it is usually a more difficult task to determine the mutual information between the input variables and the output variable than for classification problems. This paper presents a modified approach for variable selection for continuous variables adapted from a previous approach for classification problems, making use of a mutual information estimator based on the k-nearest neighbors.
Herrera, L. J., Pomares, H., Rojas, I., Verleysen, M., & Guilen, A. (2006). Effective input variable selection for function approximation. Lecture Notes in Computer Science, 4131, 41-50. https://doi.org/10.1007/11840817_5 (Original work published 2006)