Computer Science – Information Theory
Scientific paper
2007-10-05
Computer Science
Information Theory
Submitted to IEEE Transactions on Information Theory
Scientific paper
Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential entropies. A general result for the desired differential entropy convergence is provided, taking into account both compactly and uncompactly supported densities. Convergence of differential entropy is also characterized in terms of the Kullback-Liebler discriminant for densities with fairly general supports, and it is shown that convergence in variation of probability measures guarantees such convergence under an appropriate boundedness condition on the densities involved. Results for the discrete setting are also provided, allowing for infinitely supported probability measures, by taking advantage of the equivalence between weak convergence and convergence in variation in this setting.
Parada Patricio
Piera Francisco J.
No associations
LandOfFree
On Convergence Properties of Shannon Entropy does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with On Convergence Properties of Shannon Entropy, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and On Convergence Properties of Shannon Entropy will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-329548