Distribution of Mutual Information

Details Distribution of Mutual Information Distribution of Mutual Information

2001-12-15

arxiv.org/abs/cs/0112019v1

Advances in Neural Information Processing Systems 14 (NIPS-2001) pages 399-406

Computer Science

Artificial Intelligence

8 pages

Scientific paper

The mutual information of two random variables i and j with joint probabilities t_ij is commonly used in learning Bayesian nets as well as in many other fields. The chances t_ij are usually estimated by the empirical sampling frequency n_ij/n leading to a point estimate I(n_ij/n) for the mutual information. To answer questions like "is I(n_ij/n) consistent with zero?" or "what is the probability that the true mutual information is much larger than the point estimate?" one has to go beyond the point estimate. In the Bayesian framework one can answer these questions by utilizing a (second order) prior distribution p(t) comprising prior information about t. From the prior p(t) one can compute the posterior p(t|n), from which the distribution p(I|n) of the mutual information can be calculated. We derive reliable and quickly computable approximations for p(I|n). We concentrate on the mean, variance, skewness, and kurtosis, and non-informative priors. For the mean we also give an exact expression. Numerical issues and the range of validity are discussed.

Affiliated with

Also associated with

No associations

Rating

Distribution of Mutual Information does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Distribution of Mutual Information, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Distribution of Mutual Information will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-345244