A distance for probability spaces, and long-term values in Markov Decision Processes and Repeated Games

Details A distance for probability spaces, and long-term values in Markov Decision Processes and Repeated Games A distance for probability spaces, and long-term values in Markov Decision Processes and Repeated Games

2012-02-28

arxiv.org/abs/1202.6259v1

Mathematics

Optimization and Control

Scientific paper

Given a finite set $K$, we denote by $X=\Delta(K)$ the set of probabilities on $K$ and by $Z=\Delta_f(X)$ the set of Borel probabilities on $X$ with finite support. Studying a Markov Decision Process with partial information on $K$ naturally leads to a Markov Decision Process with full information on $X$. We introduce a new metric $d_*$ on $Z$ such that the transitions become 1-Lipschitz from $(X, \|.\|_1)$ to $(Z,d_*)$. In the first part of the article, we define and prove several properties of the metric $d_*$. Especially, $d_*$ satisfies a Kantorovich-Rubinstein type duality formula and can be characterized by using disintegrations. In the second part, we characterize the limit values in several classes of "compact non expansive" Markov Decision Processes. In particular we use the metric $d_*$ to characterize the limit value in Partial Observation MDP with finitely many states and in Repeated Games with an informed controller with finite sets of states and actions. Moreover in each case we can prove the existence of a generalized notion of uniform value where we consider not only the Ces\`aro mean when the number of stages is large enough but any evaluation function $\theta \in \Delta(\N^*)$ when the impatience $I(\theta)=\sum_{t\geq 1} |\theta_{t+1}-\theta_t|$ is small enough.

Affiliated with

Also associated with

No associations

Rating

A distance for probability spaces, and long-term values in Markov Decision Processes and Repeated Games does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with A distance for probability spaces, and long-term values in Markov Decision Processes and Repeated Games, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and A distance for probability spaces, and long-term values in Markov Decision Processes and Repeated Games will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-611251