Statistics – Machine Learning
Scientific paper
2009-06-19
Statistics
Machine Learning
16 pages, 3 figures
Scientific paper
Variable selection for high-dimensional linear models has received a lot of attention lately, mostly in the context of l1-regularization. Part of the attraction is the variable selection effect: parsimonious models are obtained, which are very suitable for interpretation. In terms of predictive power, however, these regularized linear models are often slightly inferior to machine learning procedures like tree ensembles. Tree ensembles, on the other hand, lack usually a formal way of variable selection and are difficult to visualize. A Garrote-style convex penalty for trees ensembles, in particular Random Forests, is proposed. The penalty selects functional groups of nodes in the trees. These could be as simple as monotone functions of individual predictor variables. This yields a parsimonious function fit, which lends itself easily to visualization and interpretation. The predictive power is maintained at least at the same level as the original tree ensemble. A key feature of the method is that, once a tree ensemble is fitted, no further tuning parameter needs to be selected. The empirical performance is demonstrated on a wide array of datasets.
No associations
LandOfFree
Forest Garrote does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Forest Garrote, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Forest Garrote will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-401067