Computer Science – Learning
Scientific paper
2011-06-26
Journal Of Artificial Intelligence Research, Volume 19, pages 205-208, 2003
Computer Science
Learning
Scientific paper
10.1613/jair.1190
Shaping has proven to be a powerful but precarious means of improving reinforcement learning performance. Ng, Harada, and Russell (1999) proposed the potential-based shaping algorithm for adding shaping rewards in a way that guarantees the learner will learn optimal behavior. In this note, we prove certain similarities between this shaping algorithm and the initialization step required for several reinforcement learning algorithms. More specifically, we prove that a reinforcement learner with initial Q-values based on the shaping algorithm's potential function make the same updates throughout learning as a learner receiving potential-based shaping rewards. We further prove that under a broad category of policies, the behavior of these two learners are indistinguishable. The comparison provides intuition on the theoretical properties of the shaping algorithm as well as a suggestion for a simpler method for capturing the algorithm's benefit. In addition, the equivalence raises previously unaddressed issues concerning the efficiency of learning with potential-based shaping.
No associations
LandOfFree
Potential-Based Shaping and Q-Value Initialization are Equivalent does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Potential-Based Shaping and Q-Value Initialization are Equivalent, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Potential-Based Shaping and Q-Value Initialization are Equivalent will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-637767