MDL Convergence Speed for Bernoulli Sequences

Details MDL Convergence Speed for Bernoulli Sequences MDL Convergence Speed for Bernoulli Sequences

2006-02-22

arxiv.org/abs/math/0602505v1

Statistics and Computing, 16 (2006) pages 161-175

Mathematics

Statistics Theory

28 pages

Scientific paper

10.1007/s11222-006-6746-3

The Minimum Description Length principle for online sequence estimation/prediction in a proper learning setup is studied. If the underlying model class is discrete, then the total expected square loss is a particularly interesting performance measure: (a) this quantity is finitely bounded, implying convergence with probability one, and (b) it additionally specifies the convergence speed. For MDL, in general one can only have loss bounds which are finite but exponentially larger than those for Bayes mixtures. We show that this is even the case if the model class contains only Bernoulli distributions. We derive a new upper bound on the prediction error for countable Bernoulli classes. This implies a small bound (comparable to the one for Bayes mixtures) for certain important model classes. We discuss the application to Machine Learning tasks such as classification and hypothesis testing, and generalization to countable classes of i.i.d. models.

Affiliated with

Also associated with

No associations

Rating

MDL Convergence Speed for Bernoulli Sequences does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with MDL Convergence Speed for Bernoulli Sequences, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and MDL Convergence Speed for Bernoulli Sequences will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-89196