Information and Entropy

Details Information and Entropy Information and Entropy

2007-10-04

arxiv.org/abs/0710.1068v1

Physics

Data Analysis, Statistics and Probability

Presented at MaxEnt 2007, the 27th International Workshop on Bayesian Inference and Maximum Entropy Methods (July 8-13, 2007,

Scientific paper

10.1063/1.2821253

What is information? Is it physical? We argue that in a Bayesian theory the notion of information must be defined in terms of its effects on the beliefs of rational agents. Information is whatever constrains rational beliefs and therefore it is the force that induces us to change our minds. This problem of updating from a prior to a posterior probability distribution is tackled through an eliminative induction process that singles out the logarithmic relative entropy as the unique tool for inference. The resulting method of Maximum relative Entropy (ME), which is designed for updating from arbitrary priors given information in the form of arbitrary constraints, includes as special cases both MaxEnt (which allows arbitrary constraints) and Bayes' rule (which allows arbitrary priors). Thus, ME unifies the two themes of these workshops -- the Maximum Entropy and the Bayesian methods -- into a single general inference scheme that allows us to handle problems that lie beyond the reach of either of the two methods separately. I conclude with a couple of simple illustrative examples.

Affiliated with

Also associated with

No associations

Rating

Information and Entropy does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Information and Entropy, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Information and Entropy will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-563738