Variable selection for the multicategory SVM via adaptive sup-norm regularization

Details Variable selection for the multicategory SVM via adaptive sup-norm regularization Variable selection for the multicategory SVM via adaptive sup-norm regularization

2008-03-26

arxiv.org/abs/0803.3676v1

Electronic Journal of Statistics 2008, Vol. 2, 149-167

Statistics

Methodology

Published in at http://dx.doi.org/10.1214/08-EJS122 the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by t

Scientific paper

10.1214/08-EJS122

The Support Vector Machine (SVM) is a popular classification paradigm in machine learning and has achieved great success in real applications. However, the standard SVM can not select variables automatically and therefore its solution typically utilizes all the input variables without discrimination. This makes it difficult to identify important predictor variables, which is often one of the primary goals in data analysis. In this paper, we propose two novel types of regularization in the context of the multicategory SVM (MSVM) for simultaneous classification and variable selection. The MSVM generally requires estimation of multiple discriminating functions and applies the argmax rule for prediction. For each individual variable, we propose to characterize its importance by the supnorm of its coefficient vector associated with different functions, and then minimize the MSVM hinge loss function subject to a penalty on the sum of supnorms. To further improve the supnorm penalty, we propose the adaptive regularization, which allows different weights imposed on different variables according to their relative importance. Both types of regularization automate variable selection in the process of building classifiers, and lead to sparse multi-classifiers with enhanced interpretability and improved accuracy, especially for high dimensional low sample size data. One big advantage of the supnorm penalty is its easy implementation via standard linear programming. Several simulated examples and one real gene data analysis demonstrate the outstanding performance of the adaptive supnorm penalty in various data settings.

Affiliated with

Also associated with

No associations

Rating

Variable selection for the multicategory SVM via adaptive sup-norm regularization does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Variable selection for the multicategory SVM via adaptive sup-norm regularization, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Variable selection for the multicategory SVM via adaptive sup-norm regularization will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-49629