Excess entropy in natural language: present state and perspectives

Details Excess entropy in natural language: present state and perspectives Excess entropy in natural language: present state and perspectives

2011-05-06

arxiv.org/abs/1105.1306v2

Computer Science

Information Theory

12 pages; no figures

Scientific paper

We review recent progress in understanding the meaning of mutual information in natural language. Let us define words in a text as strings that occur sufficiently often. In a few previous papers, we have shown that a power-law distribution for so defined words (a.k.a. Herdan's law) is obeyed if there is a similar power-law growth of (algorithmic) mutual information between adjacent portions of texts of increasing length. Moreover, the power-law growth of information holds if texts describe a complicated infinite (algorithmically) random object in a highly repetitive way, according to an analogous power-law distribution. The described object may be immutable (like a mathematical or physical constant) or may evolve slowly in time (like cultural heritage). Here we reflect on the respective mathematical results in a less technical way. We also discuss feasibility of deciding to what extent these results apply to the actual human communication.

Affiliated with

Also associated with

No associations

Rating

Excess entropy in natural language: present state and perspectives does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Excess entropy in natural language: present state and perspectives, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Excess entropy in natural language: present state and perspectives will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-576710