Discounted continuous-time constrained Markov decision processes in Polish spaces

Details Discounted continuous-time constrained Markov decision processes in Polish spaces Discounted continuous-time constrained Markov decision processes in Polish spaces

2011-12-30

arxiv.org/abs/1201.0089v1

Annals of Applied Probability 2011, Vol. 21, No. 5, 2016-2049

Mathematics

Probability

Published in at http://dx.doi.org/10.1214/10-AAP749 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Inst

Scientific paper

10.1214/10-AAP749

This paper is devoted to studying constrained continuous-time Markov decision processes (MDPs) in the class of randomized policies depending on state histories. The transition rates may be unbounded, the reward and costs are admitted to be unbounded from above and from below, and the state and action spaces are Polish spaces. The optimality criterion to be maximized is the expected discounted rewards, and the constraints can be imposed on the expected discounted costs. First, we give conditions for the nonexplosion of underlying processes and the finiteness of the expected discounted rewards/costs. Second, using a technique of occupation measures, we prove that the constrained optimality of continuous-time MDPs can be transformed to an equivalent (optimality) problem over a class of probability measures. Based on the equivalent problem and a so-called $\bar{w}$-weak convergence of probability measures developed in this paper, we show the existence of a constrained optimal policy. Third, by providing a linear programming formulation of the equivalent problem, we show the solvability of constrained optimal policies. Finally, we use two computable examples to illustrate our main results.

Affiliated with

Also associated with

No associations

Rating

Discounted continuous-time constrained Markov decision processes in Polish spaces does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Discounted continuous-time constrained Markov decision processes in Polish spaces, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Discounted continuous-time constrained Markov decision processes in Polish spaces will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-335022