Efficient, adaptive cross-validation for tuning and comparing models, with application to drug discovery

Details Efficient, adaptive cross-validation for tuning and comparing models, with application to drug discovery Efficient, adaptive cross-validation for tuning and comparing models, with application to drug discovery

2012-02-29

arxiv.org/abs/1202.6536v1

Annals of Applied Statistics 2011, Vol. 5, No. 4, 2668-2687

Statistics

Applications

Published in at http://dx.doi.org/10.1214/11-AOAS491 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Ins

Scientific paper

10.1214/11-AOAS491

Cross-validation (CV) is widely used for tuning a model with respect to user-selected parameters and for selecting a "best" model. For example, the method of $k$-nearest neighbors requires the user to choose $k$, the number of neighbors, and a neural network has several tuning parameters controlling the network complexity. Once such parameters are optimized for a particular data set, the next step is often to compare the various optimized models and choose the method with the best predictive performance. Both tuning and model selection boil down to comparing models, either across different values of the tuning parameters or across different classes of statistical models and/or sets of explanatory variables. For multiple large sets of data, like the PubChem drug discovery cheminformatics data which motivated this work, reliable CV comparisons are computationally demanding, or even infeasible. In this paper we develop an efficient sequential methodology for model comparison based on CV. It also takes into account the randomness in CV. The number of models is reduced via an adaptive, multiplicity-adjusted sequential algorithm, where poor performers are quickly eliminated. By exploiting matching of individual observations, it is sometimes even possible to establish the statistically significant inferiority of some models with just one execution of CV.

Affiliated with

Also associated with

No associations

Rating

Efficient, adaptive cross-validation for tuning and comparing models, with application to drug discovery does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Efficient, adaptive cross-validation for tuning and comparing models, with application to drug discovery, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Efficient, adaptive cross-validation for tuning and comparing models, with application to drug discovery will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-524167