Statistical physics and practical training of soft-committee machines

Details Statistical physics and practical training of soft-committee machines Statistical physics and practical training of soft-committee machines

1998-12-11

arxiv.org/abs/cond-mat/9812197v1

Physics

Condensed Matter

Disordered Systems and Neural Networks

11 pages, 4 figures

Scientific paper

10.1007/s100510050889

Equilibrium states of large layered neural networks with differentiable activation function and a single, linear output unit are investigated using the replica formalism. The quenched free energy of a student network with a very large number of hidden units learning a rule of perfectly matching complexity is calculated analytically. The system undergoes a first order phase transition from unspecialized to specialized student configurations at a critical size of the training set. Computer simulations of learning by stochastic gradient descent from a fixed training set demonstrate that the equilibrium results describe quantitatively the plateau states which occur in practical training procedures at sufficiently small but finite learning rates.

Affiliated with

Also associated with

No associations

Rating

Statistical physics and practical training of soft-committee machines does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Statistical physics and practical training of soft-committee machines, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Statistical physics and practical training of soft-committee machines will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-38397