The Ratio Index for Budgeted Learning, with Applications

Details The Ratio Index for Budgeted Learning, with Applications The Ratio Index for Budgeted Learning, with Applications

2008-10-03

arxiv.org/abs/0810.0558v1

Computer Science

Data Structures and Algorithms

Scientific paper

In the budgeted learning problem, we are allowed to experiment on a set of alternatives (given a fixed experimentation budget) with the goal of picking a single alternative with the largest possible expected payoff. Approximation algorithms for this problem were developed by Guha and Munagala by rounding a linear program that couples the various alternatives together. In this paper we present an index for this problem, which we call the ratio index, which also guarantees a constant factor approximation. Index-based policies have the advantage that a single number (i.e. the index) can be computed for each alternative irrespective of all other alternatives, and the alternative with the highest index is experimented upon. This is analogous to the famous Gittins index for the discounted multi-armed bandit problem. The ratio index has several interesting structural properties. First, we show that it can be computed in strongly polynomial time. Second, we show that with the appropriate discount factor, the Gittins index and our ratio index are constant factor approximations of each other, and hence the Gittins index also gives a constant factor approximation to the budgeted learning problem. Finally, we show that the ratio index can be used to create an index-based policy that achieves an O(1)-approximation for the finite horizon version of the multi-armed bandit problem. Moreover, the policy does not require any knowledge of the horizon (whereas we compare its performance against an optimal strategy that is aware of the horizon). This yields the following surprising result: there is an index-based policy that achieves an O(1)-approximation for the multi-armed bandit problem, oblivious to the underlying discount factor.

Affiliated with

Also associated with

No associations

Rating

The Ratio Index for Budgeted Learning, with Applications does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with The Ratio Index for Budgeted Learning, with Applications, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and The Ratio Index for Budgeted Learning, with Applications will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-409163