Global Optimization for Value Function Approximation

Details Global Optimization for Value Function Approximation Global Optimization for Value Function Approximation

2010-06-14

arxiv.org/abs/1006.2743v1

Computer Science

Artificial Intelligence

Scientific paper

Existing value function approximation methods have been successfully used in many applications, but they often lack useful a priori error bounds. We propose a new approximate bilinear programming formulation of value function approximation, which employs global optimization. The formulation provides strong a priori guarantees on both robust and expected policy loss by minimizing specific norms of the Bellman residual. Solving a bilinear program optimally is NP-hard, but this is unavoidable because the Bellman-residual minimization itself is NP-hard. We describe and analyze both optimal and approximate algorithms for solving bilinear programs. The analysis shows that this algorithm offers a convergent generalization of approximate policy iteration. We also briefly analyze the behavior of bilinear programming algorithms under incomplete samples. Finally, we demonstrate that the proposed approach can consistently minimize the Bellman residual on simple benchmark problems.

Affiliated with

Also associated with

No associations

Rating

Global Optimization for Value Function Approximation does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Global Optimization for Value Function Approximation, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Global Optimization for Value Function Approximation will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-422730