Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers

Details Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers

2007-05-16

arxiv.org/abs/0705.2318v1

Computer Science

Learning

13 pages, 9 figures

Scientific paper

10.1143/JPSJ.76.114001

We analyze the generalization performance of a student in a model composed of nonlinear perceptrons: a true teacher, ensemble teachers, and the student. We calculate the generalization error of the student analytically or numerically using statistical mechanics in the framework of on-line learning. We treat two well-known learning rules: Hebbian learning and perceptron learning. As a result, it is proven that the nonlinear model shows qualitatively different behaviors from the linear model. Moreover, it is clarified that Hebbian learning and perceptron learning show qualitatively different behaviors from each other. In Hebbian learning, we can analytically obtain the solutions. In this case, the generalization error monotonically decreases. The steady value of the generalization error is independent of the learning rate. The larger the number of teachers is and the more variety the ensemble teachers have, the smaller the generalization error is. In perceptron learning, we have to numerically obtain the solutions. In this case, the dynamical behaviors of the generalization error are non-monotonic. The smaller the learning rate is, the larger the number of teachers is; and the more variety the ensemble teachers have, the smaller the minimum value of the generalization error is.

Affiliated with

Also associated with

No associations

Rating

Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Statistical Mechanics of Nonlinear On-line Learning for Ensemble Teachers will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-670289