Empirical Bernstein Bounds and Sample Variance Penalization

Details Empirical Bernstein Bounds and Sample Variance Penalization Empirical Bernstein Bounds and Sample Variance Penalization

2009-07-21

arxiv.org/abs/0907.3740v1

Statistics

Machine Learning

10 pages, 1 figure, Proc. Computational Learning Theory Conference (COLT 2009)

Scientific paper

We give improved constants for data dependent and variance sensitive confidence bounds, called empirical Bernstein bounds, and extend these inequalities to hold uniformly over classes of functionswhose growth function is polynomial in the sample size n. The bounds lead us to consider sample variance penalization, a novel learning method which takes into account the empirical variance of the loss function. We give conditions under which sample variance penalization is effective. In particular, we present a bound on the excess risk incurred by the method. Using this, we argue that there are situations in which the excess risk of our method is of order 1/n, while the excess risk of empirical risk minimization is of order 1/sqrt/{n}. We show some experimental results, which confirm the theory. Finally, we discuss the potential application of our results to sample compression schemes.

Affiliated with

Also associated with

No associations

Rating

Empirical Bernstein Bounds and Sample Variance Penalization does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Empirical Bernstein Bounds and Sample Variance Penalization, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Empirical Bernstein Bounds and Sample Variance Penalization will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-127696