A Method for Avoiding Bias from Feature Selection with Application to Naive Bayes Classification Models

Details A Method for Avoiding Bias from Feature Selection with Application to Naive Bayes Classification Models A Method for Avoiding Bias from Feature Selection with Application to Naive Bayes Classification Models

2007-02-20

arxiv.org/abs/math/0702591v1

Mathematics

Statistics Theory

Scientific paper

For many classification and regression problems, a large number of features are available for possible use - this is typical of DNA microarray data on gene expression, for example. Often, for computational or other reasons, only a small subset of these features are selected for use in a model, based on some simple measure such as correlation with the response variable. This procedure may introduce an optimistic bias, however, in which the response variable appears to be more predictable than it actually is, because the high correlation of the selected features with the response may be partly or wholely due to chance. We show how this bias can be avoided when using a Bayesian model for the joint distribution of features and response. The crucial insight is that even if we forget the exact values of the unselected features, we should retain, and condition on, the knowledge that their correlation with the response was too small for them to be selected. In this paper we describe how this idea can be implemented for ``naive Bayes'' models of binary data. Experiments with simulated data confirm that this method avoids bias due to feature selection. We also apply the naive Bayes model to subsets of data relating gene expression to colon cancer, and find that correcting for bias from feature selection does improve predictive performance.

Affiliated with

Also associated with

No associations

Rating

A Method for Avoiding Bias from Feature Selection with Application to Naive Bayes Classification Models does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with A Method for Avoiding Bias from Feature Selection with Application to Naive Bayes Classification Models, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and A Method for Avoiding Bias from Feature Selection with Application to Naive Bayes Classification Models will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-84313