Physics – Condensed Matter – Disordered Systems and Neural Networks
Scientific paper
1999-01-15
Physics
Condensed Matter
Disordered Systems and Neural Networks
Latex, 29 pages, 2 Encapsulated Post Script figures. To appear in Journal of Physics A
Scientific paper
10.1088/0305-4470/32/10/007
We calculate the mutual information (MI) of a two-layered neural network with noiseless, continuous inputs and binary, stochastic outputs under several assumptions on the synaptic efficiencies. The interesting regime corresponds to the limit where the number of both input and output units is large but their ratio is kept fixed at a value $\alpha$. We first present a solution for the MI using the replica technique with a replica symmetric (RS) ansatz. Then we find an exact solution for this quantity valid in a neighborhood of $\alpha = 0$. An analysis of this solution shows that the system must have a phase transition at some finite value of $\alpha$. This transition shows a singularity in the third derivative of the MI. As the RS solution turns out to be infinitely differentiable, it could be regarded as a smooth approximation to the MI. This is checked numerically in the validity domain of the exact solution.
Korutcheva Elka
Parga Nestor
Turiel Antonio
No associations
LandOfFree
The mutual information of a stochastic binary channel: validity of the Replica Symmetry Ansatz does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with The mutual information of a stochastic binary channel: validity of the Replica Symmetry Ansatz, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and The mutual information of a stochastic binary channel: validity of the Replica Symmetry Ansatz will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-597428