Statistical Mechanical Analysis of the Dynamics of Learning in Perceptrons

Details Statistical Mechanical Analysis of the Dynamics of Learning in Perceptrons Statistical Mechanical Analysis of the Dynamics of Learning in Perceptrons

1997-05-23

arxiv.org/abs/cond-mat/9705243v1

Physics

Condensed Matter

Disordered Systems and Neural Networks

56 pages LaTex with Postscript figures included in text

Scientific paper

We describe the application of tools from statistical mechanics to analyse the dynamics of various classes of supervised learning rules in perceptrons. The character of this paper is mostly that of a cross between a biased non-encyclopedic review and lecture notes: we try to present a coherent and self-contained picture of the basics of this field, to explain the ideas and tricks, to show how the predictions of the theory compare with (simulation) experiments, and to bring together scattered results. Technical details are given explicitly in an appendix. In order to avoid distraction we concentrate the references in a final section. In addition this paper contains some new results: (i) explicit solutions of the macroscopic equations that describe the error evolution for on-line and batch learning rules, (ii) an analysis of the dynamics of arbitrary macroscopic observables (for complete and incomplete trainingsets), leading to a general Fokker-Planck equation, and (iii) the macroscopic laws describing batch learning with complete training sets. We close the paper with a preliminary expose of ongoing research on the dynamics of learning for the case where the training set is incomplete (i.e. where the number of examples scales linearly with the network size).

Affiliated with

Also associated with

No associations

Rating

Statistical Mechanical Analysis of the Dynamics of Learning in Perceptrons does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Statistical Mechanical Analysis of the Dynamics of Learning in Perceptrons, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Statistical Mechanical Analysis of the Dynamics of Learning in Perceptrons will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-205192