Weight space structure and analysis using a finite replica number in the Ising perceptron

Details Weight space structure and analysis using a finite replica number in the Ising perceptron Weight space structure and analysis using a finite replica number in the Ising perceptron

2009-10-13

arxiv.org/abs/0910.2281v2

J. Stat. Mech. (2009) P12014

Physics

Condensed Matter

Disordered Systems and Neural Networks

21 pages, 11 figures, Added references, some comments, and corrections to minor errors

Scientific paper

The weight space of the Ising perceptron in which a set of random patterns is stored is examined using the generating function of the partition function $\phi(n)=(1/N)\log [Z^n]$ as the dimension of the weight vector $N$ tends to infinity, where $Z$ is the partition function and $[ ... ]$ represents the configurational average. We utilize $\phi(n)$ for two purposes, depending on the value of the ratio $\alpha=M/N$, where $M$ is the number of random patterns. For $\alpha < \alpha_{\rm s}=0.833 ...$, we employ $\phi(n)$, in conjunction with Parisi's one-step replica symmetry breaking scheme in the limit of $n \to 0$, to evaluate the complexity that characterizes the number of disjoint clusters of weights that are compatible with a given set of random patterns, which indicates that, in typical cases, the weight space is equally dominated by a single large cluster of exponentially many weights and exponentially many small clusters of a single weight. For $\alpha > \alpha_{\rm s}$, on the other hand, $\phi(n)$ is used to assess the rate function of a small probability that a given set of random patterns is atypically separable by the Ising perceptrons. We show that the analyticity of the rate function changes at $\alpha = \alpha_{\rm GD}=1.245 ... $, which implies that the dominant configuration of the atypically separable patterns exhibits a phase transition at this critical ratio. Extensive numerical experiments are conducted to support the theoretical predictions.

Affiliated with

Also associated with

No associations

Rating

Weight space structure and analysis using a finite replica number in the Ising perceptron does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Weight space structure and analysis using a finite replica number in the Ising perceptron, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Weight space structure and analysis using a finite replica number in the Ising perceptron will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-640083