Dynamics of Learning with Restricted Training Sets I: General Theory

Details Dynamics of Learning with Restricted Training Sets I: General Theory Dynamics of Learning with Restricted Training Sets I: General Theory

1999-09-28

arxiv.org/abs/cond-mat/9909402v1

Physics

Condensed Matter

Disordered Systems and Neural Networks

39 pages, LaTeX

Scientific paper

10.1103/PhysRevE.62.5444

We study the dynamics of supervised learning in layered neural networks, in the regime where the size $p$ of the training set is proportional to the number $N$ of inputs. Here the local fields are no longer described by Gaussian probability distributions and the learning dynamics is of a spin-glass nature, with the composition of the training set playing the role of quenched disorder. We show how dynamical replica theory can be used to predict the evolution of macroscopic observables, including the two relevant performance measures (training error and generalization error), incorporating the old formalism developed for complete training sets in the limit $\alpha=p/N\to\infty$ as a special case. For simplicity we restrict ourselves in this paper to single-layer networks and realizable tasks.

Affiliated with

Also associated with

No associations

Rating

Dynamics of Learning with Restricted Training Sets I: General Theory does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Dynamics of Learning with Restricted Training Sets I: General Theory, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Dynamics of Learning with Restricted Training Sets I: General Theory will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-423660