Gradient descent learning in and out of equilibrium

Details Gradient descent learning in and out of equilibrium Gradient descent learning in and out of equilibrium

2000-04-04

arxiv.org/abs/cond-mat/0004047v1

Physics

Condensed Matter

Disordered Systems and Neural Networks

08 pages, submitted to the Journal of Physics A

Scientific paper

10.1103/PhysRevE.63.061905

Relations between the off thermal equilibrium dynamical process of on-line learning and the thermally equilibrated off-line learning are studied for potential gradient descent learning. The approach of Opper to study on-line Bayesian algorithms is extended to potential based or maximum likelihood learning. We look at the on-line learning algorithm that best approximates the off-line algorithm in the sense of least Kullback-Leibler information loss. It works by updating the weights along the gradient of an effective potential different from the parent off-line potential. The interpretation of this off equilibrium dynamics holds some similarities to the cavity approach of Griniasty. We are able to analyze networks with non-smooth transfer functions and transfer the smoothness requirement to the potential.

Affiliated with

Also associated with

No associations

Rating

Gradient descent learning in and out of equilibrium does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Gradient descent learning in and out of equilibrium, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Gradient descent learning in and out of equilibrium will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-72932