The Cavity Approach to Noisy Learning in Nonlinear Perceptrons

Details The Cavity Approach to Noisy Learning in Nonlinear Perceptrons The Cavity Approach to Noisy Learning in Nonlinear Perceptrons

2001-07-19

arxiv.org/abs/cond-mat/0107405v1

Physics

Condensed Matter

Disordered Systems and Neural Networks

13 figures, one table

Scientific paper

10.1103/PhysRevE.64.061912

We analyze the learning of noisy teacher-generated examples by nonlinear and differentiable student perceptrons using the cavity method. The generic activation of an example is a function of the cavity activation of the example, which is its activation in the perceptron that learns without the example. Mean field equations for the macroscopic parameters and the stability condition yield results consistent with the replica method. When a single value of the cavity activation maps to multiple values of the generic activation, there is a competition in learning strategy between preferentially learning an example and sacrificing it in favor of the background adjustment. We find parameter regimes in which examples are learned preferentially or sacrificially, leading to a gap in the activation distribution. Full phase diagrams of this complex system are presented, and the theory predicts the existence of a phase transition from poor to good generalization states in the system. Simulation results confirm the theoretical predictions.

Affiliated with

Also associated with

No associations

Rating

The Cavity Approach to Noisy Learning in Nonlinear Perceptrons does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with The Cavity Approach to Noisy Learning in Nonlinear Perceptrons, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and The Cavity Approach to Noisy Learning in Nonlinear Perceptrons will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-657399