MDP Optimal Control under Temporal Logic Constraints

Details MDP Optimal Control under Temporal Logic Constraints MDP Optimal Control under Temporal Logic Constraints

2011-03-22

arxiv.org/abs/1103.4342v2

Computer Science

Robotics

Technical report accompanying the CDC2011 submission

Scientific paper

In this paper, we develop a method to automatically generate a control policy for a dynamical system modeled as a Markov Decision Process (MDP). The control specification is given as a Linear Temporal Logic (LTL) formula over a set of propositions defined on the states of the MDP. We synthesize a control policy such that the MDP satisfies the given specification almost surely, if such a policy exists. In addition, we designate an "optimizing proposition" to be repeatedly satisfied, and we formulate a novel optimization criterion in terms of minimizing the expected cost in between satisfactions of this proposition. We propose a sufficient condition for a policy to be optimal, and develop a dynamic programming algorithm that synthesizes a policy that is optimal under some conditions, and sub-optimal otherwise. This problem is motivated by robotic applications requiring persistent tasks, such as environmental monitoring or data gathering, to be performed.

Affiliated with

Also associated with

No associations

Rating

MDP Optimal Control under Temporal Logic Constraints does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with MDP Optimal Control under Temporal Logic Constraints, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and MDP Optimal Control under Temporal Logic Constraints will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-49595