Learning Curves for Mutual Information Maximization

Details Learning Curves for Mutual Information Maximization Learning Curves for Mutual Information Maximization

2003-05-12

arxiv.org/abs/cond-mat/0305254v1

Physics

Condensed Matter

Disordered Systems and Neural Networks

13 pages, to appear in Phys.Rev. E

Scientific paper

10.1103/PhysRevE.68.016106

An unsupervised learning procedure based on maximizing the mutual information between the outputs of two networks receiving different but statistically dependent inputs is analyzed (Becker and Hinton, Nature, 355, 92, 161). For a generic data model, I show that in the large sample limit the structure in the data is recognized by mutual information maximization. For a more restricted model, where the networks are similar to perceptrons, I calculate the learning curves for zero-temperature Gibbs learning. These show that convergence can be rather slow, and a way of regularizing the procedure is considered.

Affiliated with

Also associated with

No associations

Rating

Learning Curves for Mutual Information Maximization does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Learning Curves for Mutual Information Maximization, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Learning Curves for Mutual Information Maximization will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-202211