Physics – Condensed Matter – Disordered Systems and Neural Networks
Scientific paper
2004-01-30
Physics
Condensed Matter
Disordered Systems and Neural Networks
10 pages, 4 figures
Scientific paper
In this paper we present a method which assigns to each layer of a multilayer neural network, whose network dynamics is governed by a noisy winner-take-all mechanism, a neural temperature. This neural temperature is obtained by a least mean square fit of the probability distribution of the noisy winner-take-all mechanism to the distribution of a softmax mechanism, which has a well defined temperature as free parameter. We call this approximated temperature resulting from the optimization step the neural temperature. We apply this method to a multilayer neural network during learning the XOR-problem with a Hebb-like learning rule and show that after a transient the neural temperature decreases in each layer according to a power law. This indicates a self-organized annealing behavior induced by the learning rule itself instead of an external adjustment of a control parameter as in physically motivated optimization methods e.g. simulated annealing.
No associations
LandOfFree
Self-organized annealing in laterally inhibited neural networks shows power law decay does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Self-organized annealing in laterally inhibited neural networks shows power law decay, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Self-organized annealing in laterally inhibited neural networks shows power law decay will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-628499