Supervised Learning with Restricted Training Sets: a Generating Functional Analysis

Details Supervised Learning with Restricted Training Sets: a Generating Functional Analysis Supervised Learning with Restricted Training Sets: a Generating Functional Analysis

2001-02-15

arxiv.org/abs/cond-mat/0102272v1

Physics

Condensed Matter

Disordered Systems and Neural Networks

16 pages, 5 figures, uses iopart class

Scientific paper

10.1088/0305-4470/34/42/318

We study the dynamics of supervised on-line learning of realizable tasks in feed-forward neural networks. We focus on the regime where the number of examples used for training is proportional to the number of input channels N. Using generating function techniques from spin glass theory, we are able to average over the composition of the training set and transform the problem for N to infinity to an effective single pattern system, described completely by the student autocovariance, the student-teacher overlap and the student response function, with exact closed equations. Our method applies to arbitrary learning rules, i.e. not necessarily of a gradient-descent type. The resulting exact macroscopic dynamical equations can be integrated without finite-size effects up to any degree of accuracy, but their main value is in providing an exact and simple starting point for analytical approximation schemes. Finally, we show how, in the region of absent anomalous response and using the hypothesis that (as in detailed balance systems) the short-time part of the various operators can be transformed away, one can describe the stationary state of the network succesfully by a set of coupled equations involving only four scalar order parameters.

Affiliated with

Also associated with

No associations

Rating

Supervised Learning with Restricted Training Sets: a Generating Functional Analysis does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Supervised Learning with Restricted Training Sets: a Generating Functional Analysis, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Supervised Learning with Restricted Training Sets: a Generating Functional Analysis will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-202695