Certain Relations between Mutual Information and Fidelity of Statistical Estimation

Details Certain Relations between Mutual Information and Fidelity of Statistical Estimation Certain Relations between Mutual Information and Fidelity of Statistical Estimation

2010-10-07

arxiv.org/abs/1010.1508v1

Statistics

Applications

Scientific paper

I present several new relations between mutual information (MI) and statistical estimation error for a system that can be regarded simultaneously as a communication channel and as an estimator of an input parameter. I first derive a second-order result between MI and Fisher information (FI) that is valid for sufficiently narrow priors, but arbitrary channels. A second relation furnishes a lower bound on the MI in terms of the minimum mean-squared error (MMSE) on the Bayesian estimation of the input parameter from the channel output, one that is valid for arbitrary channels and priors. The existence of such a lower bound, while extending previous work relating the MI to the FI that is valid only in the asymptotic and high-SNR limits, elucidates further the fundamental connection between information and estimation theoretic measures of fidelity. The remaining relations I present are inequalities and correspondences among MI, FI, and MMSE in the presence of nuisance parameters.

Affiliated with

Also associated with

No associations

Rating

Certain Relations between Mutual Information and Fidelity of Statistical Estimation does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Certain Relations between Mutual Information and Fidelity of Statistical Estimation, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Certain Relations between Mutual Information and Fidelity of Statistical Estimation will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-510047