Combinatorial Entropies and Statistics

Details Combinatorial Entropies and Statistics Combinatorial Entropies and Statistics

2009-02-18

arxiv.org/abs/0902.3038v2

Physics

Condensed Matter

Statistical Mechanics

Invited contribution to the SigmaPhi 2008 Conference; accepted by EPJB volume 69 issue 3 June 2009

Scientific paper

We examine the {combinatorial} or {probabilistic} definition ("Boltzmann's principle") of the entropy or cross-entropy function $H \propto \ln \mathbb{W}$ or $D \propto - \ln \mathbb{P}$, where $\mathbb{W}$ is the statistical weight and $\mathbb{P}$ the probability of a given realization of a system. Extremisation of $H$ or $D$, subject to any constraints, thus selects the "most probable" (MaxProb) realization. If the system is multinomial, $D$ converges asymptotically (for number of entities $N \back \to \back \infty$) to the Kullback-Leibler cross-entropy $D_{KL}$; for equiprobable categories in a system, $H$ converges to the Shannon entropy $H_{Sh}$. However, in many cases $\mathbb{W}$ or $\mathbb{P}$ is not multinomial and/or does not satisfy an asymptotic limit. Such systems cannot meaningfully be analysed with $D_{KL}$ or $H_{Sh}$, but can be analysed directly by MaxProb. This study reviews several examples, including (a) non-asymptotic systems; (b) systems with indistinguishable entities (quantum statistics); (c) systems with indistinguishable categories; (d) systems represented by urn models, such as "neither independent nor identically distributed" (ninid) sampling; and (e) systems representable in graphical form, such as decision trees and networks. Boltzmann's combinatorial definition of entropy is shown to be of greater importance for {"probabilistic inference"} than the axiomatic definition used in information theory.

Affiliated with

Also associated with

No associations

Rating

Combinatorial Entropies and Statistics does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Combinatorial Entropies and Statistics, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Combinatorial Entropies and Statistics will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-127764