On the Dual Formulation of Boosting Algorithms

Details On the Dual Formulation of Boosting Algorithms On the Dual Formulation of Boosting Algorithms

2009-01-23

arxiv.org/abs/0901.3590v4

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010

Computer Science

Learning

16 pages. To publish/Published in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010

Scientific paper

We study boosting algorithms from a new perspective. We show that the Lagrange dual problems of AdaBoost, LogitBoost and soft-margin LPBoost with generalized hinge loss are all entropy maximization problems. By looking at the dual problems of these boosting algorithms, we show that the success of boosting algorithms can be understood in terms of maintaining a better margin distribution by maximizing margins and at the same time controlling the margin variance.We also theoretically prove that, approximately, AdaBoost maximizes the average margin, instead of the minimum margin. The duality formulation also enables us to develop column generation based optimization algorithms, which are totally corrective. We show that they exhibit almost identical classification results to that of standard stage-wise additive boosting algorithms but with much faster convergence rates. Therefore fewer weak classifiers are needed to build the ensemble using our proposed optimization technique.

Affiliated with

Also associated with

No associations

Rating

On the Dual Formulation of Boosting Algorithms does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with On the Dual Formulation of Boosting Algorithms, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and On the Dual Formulation of Boosting Algorithms will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-65480