Analysis of ensemble learning using simple perceptrons based on online learning theory

Details Analysis of ensemble learning using simple perceptrons based on online learning theory Analysis of ensemble learning using simple perceptrons based on online learning theory

2004-03-25

arxiv.org/abs/cond-mat/0403632v3

Physics

Condensed Matter

Disordered Systems and Neural Networks

30 pages, 17 figures

Scientific paper

10.1103/PhysRevE.71.036116

Ensemble learning of $K$ nonlinear perceptrons, which determine their outputs by sign functions, is discussed within the framework of online learning and statistical mechanics. One purpose of statistical learning theory is to theoretically obtain the generalization error. This paper shows that ensemble generalization error can be calculated by using two order parameters, that is, the similarity between a teacher and a student, and the similarity among students. The differential equations that describe the dynamical behaviors of these order parameters are derived in the case of general learning rules. The concrete forms of these differential equations are derived analytically in the cases of three well-known rules: Hebbian learning, perceptron learning and AdaTron learning. Ensemble generalization errors of these three rules are calculated by using the results determined by solving their differential equations. As a result, these three rules show different characteristics in their affinity for ensemble learning, that is ``maintaining variety among students." Results show that AdaTron learning is superior to the other two rules with respect to that affinity.

Affiliated with

Also associated with

No associations

Rating

Analysis of ensemble learning using simple perceptrons based on online learning theory does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Analysis of ensemble learning using simple perceptrons based on online learning theory, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Analysis of ensemble learning using simple perceptrons based on online learning theory will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-539418