Finding Approximate POMDP solutions Through Belief Compression

Details Finding Approximate POMDP solutions Through Belief Compression Finding Approximate POMDP solutions Through Belief Compression

2011-06-30

arxiv.org/abs/1107.0053v2

Journal Of Artificial Intelligence Research, Volume 23, pages 1-40, 2005

Computer Science

Artificial Intelligence

Scientific paper

10.1613/jair.1496

Standard value function approaches to finding policies for Partially Observable Markov Decision Processes (POMDPs) are generally considered to be intractable for large models. The intractability of these algorithms is to a large extent a consequence of computing an exact, optimal policy over the entire belief space. However, in real-world POMDP problems, computing the optimal policy for the full belief space is often unnecessary for good control even for problems with complicated policy classes. The beliefs experienced by the controller often lie near a structured, low-dimensional subspace embedded in the high-dimensional belief space. Finding a good approximation to the optimal value function for only this subspace can be much easier than computing the full value function. We introduce a new method for solving large-scale POMDPs by reducing the dimensionality of the belief space. We use Exponential family Principal Components Analysis (Collins, Dasgupta and Schapire, 2002) to represent sparse, high-dimensional belief spaces using small sets of learned features of the belief state. We then plan only in terms of the low-dimensional belief features. By planning in this low-dimensional space, we can find policies for POMDP models that are orders of magnitude larger than models that can be handled by conventional techniques. We demonstrate the use of this algorithm on a synthetic problem and on mobile robot navigation tasks.

Affiliated with

Also associated with

No associations

Rating

Finding Approximate POMDP solutions Through Belief Compression does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Finding Approximate POMDP solutions Through Belief Compression, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Finding Approximate POMDP solutions Through Belief Compression will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-480040