Origins of the Combinatorial Basis of Entropy

Details Origins of the Combinatorial Basis of Entropy Origins of the Combinatorial Basis of Entropy

2007-08-14

arxiv.org/abs/0708.1861v3

Physics

Classical Physics

MaxEnt07 manuscript, version 4 revised

Scientific paper

10.1063/1.2821255

The combinatorial basis of entropy, given by Boltzmann, can be written $H = N^{-1} \ln \mathbb{W}$, where $H$ is the dimensionless entropy, $N$ is the number of entities and $\mathbb{W}$ is number of ways in which a given realization of a system can occur (its statistical weight). This can be broadened to give generalized combinatorial (or probabilistic) definitions of entropy and cross-entropy: $H=\kappa (\phi(\mathbb{W}) +C)$ and $D=-\kappa (\phi(\mathbb{P}) +C)$, where $\mathbb{P}$ is the probability of a given realization, $\phi$ is a convenient transformation function, $\kappa$ is a scaling parameter and $C$ an arbitrary constant. If $\mathbb{W}$ or $\mathbb{P}$ satisfy the multinomial weight or distribution, then using $\phi(\cdot)=\ln(\cdot)$ and $\kappa=N^{-1}$, $H$ and $D$ asymptotically converge to the Shannon and Kullback-Leibler functions. In general, however, $\mathbb{W}$ or $\mathbb{P}$ need not be multinomial, nor may they approach an asymptotic limit. In such cases, the entropy or cross-entropy function can be {\it defined} so that its extremization ("MaxEnt'' or "MinXEnt"), subject to the constraints, gives the ``most probable'' (``MaxProb'') realization of the system. This gives a probabilistic basis for MaxEnt and MinXEnt, independent of any information-theoretic justification. This work examines the origins of the governing distribution $\mathbb{P}$.... (truncated)

Affiliated with

Also associated with

No associations

Rating

Origins of the Combinatorial Basis of Entropy does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Origins of the Combinatorial Basis of Entropy, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Origins of the Combinatorial Basis of Entropy will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-389452