PAC-Bayesian Analysis of Martingales and Multiarmed Bandits

Details PAC-Bayesian Analysis of Martingales and Multiarmed Bandits PAC-Bayesian Analysis of Martingales and Multiarmed Bandits

2011-05-12

arxiv.org/abs/1105.2416v2

Computer Science

Learning

Scientific paper

We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This lemma provides an alternative tool to Hoeffding-Azuma inequality to bound concentration of martingale values. Our second approach is based on integration of Hoeffding-Azuma inequality with PAC-Bayesian analysis. We also introduce a way to apply PAC-Bayesian analysis in situation of limited feedback. We combine the new tools to derive PAC-Bayesian generalization and regret bounds for the multiarmed bandit problem. Although our regret bound is not yet as tight as state-of-the-art regret bounds based on other well-established techniques, our results significantly expand the range of potential applications of PAC-Bayesian analysis and introduce a new analysis tool to reinforcement learning and many other fields, where martingales and limited feedback are encountered.

Affiliated with

Also associated with

No associations

Rating

PAC-Bayesian Analysis of Martingales and Multiarmed Bandits does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with PAC-Bayesian Analysis of Martingales and Multiarmed Bandits, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and PAC-Bayesian Analysis of Martingales and Multiarmed Bandits will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-496791