Fast Learning Rate of Non-Sparse Multiple Kernel Learning and Optimal Regularization Strategies

Details Fast Learning Rate of Non-Sparse Multiple Kernel Learning and Optimal Regularization Strategies Fast Learning Rate of Non-Sparse Multiple Kernel Learning and Optimal Regularization Strategies

2011-11-16

arxiv.org/abs/1111.3781v1

Statistics

Machine Learning

45 pages, 5 figures. The short version is accepted by NIPS2011

Scientific paper

In this paper, we give a new generalization error bound of Multiple Kernel Learning (MKL) for a general class of regularizations, and discuss what kind of regularization gives a favorable predictive accuracy. Our main target in this paper is dense type regularizations including \ellp-MKL. According to the recent numerical experiments, the sparse regularization does not necessarily show a good performance compared with dense type regularizations. Motivated by this fact, this paper gives a general theoretical tool to derive fast learning rates of MKL that is applicable to arbitrary mixed-norm-type regularizations in a unifying manner. This enables us to compare the generalization performances of various types of regularizations. As a consequence, we observe that the homogeneity of the complexities of candidate reproducing kernel Hilbert spaces (RKHSs) affects which regularization strategy (\ell1 or dense) is preferred. In fact, in homogeneous complexity settings where the complexities of all RKHSs are evenly same, \ell1-regularization is optimal among all isotropic norms. On the other hand, in inhomogeneous complexity settings, dense type regularizations can show better learning rate than sparse \ell1-regularization. We also show that our learning rate achieves the minimax lower bound in homogeneous complexity settings.

Affiliated with

Also associated with

No associations

Rating

Fast Learning Rate of Non-Sparse Multiple Kernel Learning and Optimal Regularization Strategies does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Fast Learning Rate of Non-Sparse Multiple Kernel Learning and Optimal Regularization Strategies, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Fast Learning Rate of Non-Sparse Multiple Kernel Learning and Optimal Regularization Strategies will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-68347