Average optimality for continuous-time Markov decision processes in polish spaces

Details Average optimality for continuous-time Markov decision processes in polish spaces Average optimality for continuous-time Markov decision processes in polish spaces

2006-07-04

arxiv.org/abs/math/0607098v1

Annals of Applied Probability 2006, Vol. 16, No. 2, 730-756

Mathematics

Probability

Published at http://dx.doi.org/10.1214/105051606000000105 in the Annals of Applied Probability (http://www.imstat.org/aap/) by

Scientific paper

10.1214/105051606000000105

This paper is devoted to studying the average optimality in continuous-time Markov decision processes with fairly general state and action spaces. The criterion to be maximized is expected average rewards. The transition rates of underlying continuous-time jump Markov processes are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. We first provide two optimality inequalities with opposed directions, and also give suitable conditions under which the existence of solutions to the two optimality inequalities is ensured. Then, from the two optimality inequalities we prove the existence of optimal (deterministic) stationary policies by using the Dynkin formula. Moreover, we present a ``semimartingale characterization'' of an optimal stationary policy. Finally, we use a generalized Potlach process with control to illustrate the difference between our conditions and those in the previous literature, and then further apply our results to average optimal control problems of generalized birth--death systems, upwardly skip-free processes and two queueing systems. The approach developed in this paper is slightly different from the ``optimality inequality approach'' widely used in the previous literature.

Affiliated with

Also associated with

No associations

Rating

Average optimality for continuous-time Markov decision processes in polish spaces does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Average optimality for continuous-time Markov decision processes in polish spaces, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Average optimality for continuous-time Markov decision processes in polish spaces will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-240156