Refinements of Universal Approximation Results for Deep Belief Networks and Restricted Boltzmann Machines

Details Refinements of Universal Approximation Results for Deep Belief Networks and Restricted Boltzmann Machines Refinements of Universal Approximation Results for Deep Belief Networks and Restricted Boltzmann Machines

2010-05-10

arxiv.org/abs/1005.1593v2

Statistics

Machine Learning

Scientific paper

We improve recently published results about resources of Restricted Boltzmann Machines (RBM) and Deep Belief Networks (DBN) required to make them Universal Approximators. We show that any distribution p on the set of binary vectors of length n can be arbitrarily well approximated by an RBM with k-1 hidden units, where k is the minimal number of pairs of binary vectors differing in only one entry such that their union contains the support set of p. In important cases this number is half of the cardinality of the support set of p. We construct a DBN with 2^n/2(n-b), b ~ log(n), hidden layers of width n that is capable of approximating any distribution on {0,1}^n arbitrarily well. This confirms a conjecture presented by Le Roux and Bengio 2010.

Affiliated with

Also associated with

No associations

Rating

Refinements of Universal Approximation Results for Deep Belief Networks and Restricted Boltzmann Machines does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Refinements of Universal Approximation Results for Deep Belief Networks and Restricted Boltzmann Machines, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Refinements of Universal Approximation Results for Deep Belief Networks and Restricted Boltzmann Machines will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-628569