Mean field for Markov Decision Processes: from Discrete to Continuous Optimization

Details Mean field for Markov Decision Processes: from Discrete to Continuous Optimization Mean field for Markov Decision Processes: from Discrete to Continuous Optimization

2010-04-14

arxiv.org/abs/1004.2342v3

Computer Science

Artificial Intelligence

Scientific paper

We study the convergence of Markov Decision Processes made of a large number of objects to optimization problems on ordinary differential equations (ODE). We show that the optimal reward of such a Markov Decision Process, satisfying a Bellman equation, converges to the solution of a continuous Hamilton-Jacobi-Bellman (HJB) equation based on the mean field approximation of the Markov Decision Process. We give bounds on the difference of the rewards, and a constructive algorithm for deriving an approximating solution to the Markov Decision Process from a solution of the HJB equations. We illustrate the method on three examples pertaining respectively to investment strategies, population dynamics control and scheduling in queues are developed. They are used to illustrate and justify the construction of the controlled ODE and to show the gain obtained by solving a continuous HJB equation rather than a large discrete Bellman equation.

Affiliated with

Also associated with

No associations

Rating

Mean field for Markov Decision Processes: from Discrete to Continuous Optimization does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Mean field for Markov Decision Processes: from Discrete to Continuous Optimization, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Mean field for Markov Decision Processes: from Discrete to Continuous Optimization will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-525654