Identification of Statistically Significant Features from Random Forests
(2013) ECML workshop on Solving Complex Machine Learning Problems with Ensemble Methods — Location: Prague (Czech Republic) (27.September.2013)
Files
2013COPEM_paper.pdf
Restricted Access - Adobe PDF
- 552.48 KB
Details
- Authors
- Abstract
- Embedded feature selection can be performed by analyzing the variables used in a Random Forest. Such a multivariate selection takes into account the interactions between variables but is not easy to interpret in a statistical sense. We propose a statistical procedure to measure variable importance that tests if variables are significantly useful in combination with others in a forest. We show experimentally that this new importance index correctly identifies relevant variables. The top of the variable ranking is, as expected, largely correlated with Breiman’s importance index based on a permutation test. Our measure has the additional benefit to produce p-values from the forest voting process. Such p-values offer a very natural way to decide which features are significantly relevant while controlling the false discovery rate.
- Affiliations
Citations
Paul, J., Verleysen, M., & Dupont, P. (2013). Identification of Statistically Significant Features from Random Forests. ECML workshop on Solving Complex Machine Learning Problems with Ensemble Methods, Prague (Czech Republic). https://hdl.handle.net/2078.5/253855