A Unifying Framework for Linearly Solvable Control

Details A Unifying Framework for Linearly Solvable Control A Unifying Framework for Linearly Solvable Control

2012-02-14

arxiv.org/abs/1202.3715v1

Computer Science

Systems and Control

Scientific paper

Recent work has led to the development of an elegant theory of Linearly Solvable Markov Decision Processes (LMDPs) and related Path-Integral Control Problems. Traditionally, MDPs have been formulated using stochastic policies and a control cost based on the KL divergence. In this paper, we extend this framework to a more general class of divergences: the Renyi divergences. These are a more general class of divergences parameterized by a continuous parameter that include the KL divergence as a special case. The resulting control problems can be interpreted as solving a risk-sensitive version of the LMDP problem. For a > 0, we get risk-averse behavior (the degree of risk-aversion increases with a) and for a < 0, we get risk-seeking behavior. We recover LMDPs in the limit as a -> 0. This work generalizes the recently developed risk-sensitive path-integral control formalism which can be seen as the continuous-time limit of results obtained in this paper. To the best of our knowledge, this is a general theory of linearly solvable control and includes all previous work as a special case. We also present an alternative interpretation of these results as solving a 2-player (cooperative or competitive) Markov Game. From the linearity follow a number of nice properties including compositionality of control laws and a path-integral representation of the value function. We demonstrate the usefulness of the framework on control problems with noise where different values of lead to qualitatively different control behaviors.

Affiliated with

Also associated with

No associations

Rating

A Unifying Framework for Linearly Solvable Control does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with A Unifying Framework for Linearly Solvable Control, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and A Unifying Framework for Linearly Solvable Control will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-90408