Computer Science – Information Theory
Scientific paper
2008-09-05
Computer Science
Information Theory
A somewhat modified version of this paper was published in Statistica Neerlandica 62(3), pages 374-392, 2008
Scientific paper
We give a characterization of Maximum Entropy/Minimum Relative Entropy inference by providing two `strong entropy concentration' theorems. These theorems unify and generalize Jaynes' `concentration phenomenon' and Van Campenhout and Cover's `conditional limit theorem'. The theorems characterize exactly in what sense a prior distribution Q conditioned on a given constraint, and the distribution P, minimizing the relative entropy D(P ||Q) over all distributions satisfying the constraint, are `close' to each other. We then apply our theorems to establish the relationship between entropy concentration and a game-theoretic characterization of Maximum Entropy Inference due to Topsoe and others.
No associations
LandOfFree
Entropy Concentration and the Empirical Coding Game does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Entropy Concentration and the Empirical Coding Game, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Entropy Concentration and the Empirical Coding Game will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-659036