Estimation of Rényi Entropy and Mutual Information Based on Generalized Nearest-Neighbor Graphs

Details Estimation of Rényi Entropy and Mutual Information Based on Generalized Nearest-Neighbor Graphs Estimation of Rényi Entropy and Mutual Information Based on Generalized Nearest-Neighbor Graphs

2010-03-09

arxiv.org/abs/1003.1954v2

Statistics

Machine Learning

to appear at NIPS 2010 (Neural Information Processing Systems)

Scientific paper

We present simple and computationally efficient nonparametric estimators of R\'enyi entropy and mutual information based on an i.i.d. sample drawn from an unknown, absolutely continuous distribution over $\R^d$. The estimators are calculated as the sum of $p$-th powers of the Euclidean lengths of the edges of the `generalized nearest-neighbor' graph of the sample and the empirical copula of the sample respectively. For the first time, we prove the almost sure consistency of these estimators and upper bounds on their rates of convergence, the latter of which under the assumption that the density underlying the sample is Lipschitz continuous. Experiments demonstrate their usefulness in independent subspace analysis.

Affiliated with

Also associated with

No associations

Rating

Estimation of Rényi Entropy and Mutual Information Based on Generalized Nearest-Neighbor Graphs does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Estimation of Rényi Entropy and Mutual Information Based on Generalized Nearest-Neighbor Graphs, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Estimation of Rényi Entropy and Mutual Information Based on Generalized Nearest-Neighbor Graphs will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-455777