Physics – Data Analysis – Statistics and Probability
Scientific paper
2012-02-14
Physics
Data Analysis, Statistics and Probability
6 pages, 4 figures
Scientific paper
Zipf's law on word frequency is observed in English, French, Spanish, Italian, and so on, yet it does not hold for Chinese, Japanese or Korean characters. A model for writing process is proposed to explain the above difference, which takes into account the effects of finite vocabulary size. Experiments, simulations and analytical solution agree well with each other. The results show that the frequency distribution follows a power law with exponent being equal to 1, at which the corresponding Zipf's exponent diverges. Actually, the distribution obeys exponential form in the Zipf's plot. Deviating from the Heaps' law, the number of distinct words grows with the text length in three stages: It grows linearly in the beginning, then turns to a logarithmical form, and eventually saturates. This work refines previous understanding about Zipf's law and Heaps' law in language systems.
Lu Linyuan
Zhang Zi-Ke
Zhou Tianchun
No associations
LandOfFree
Scaling Laws in Human Language does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Scaling Laws in Human Language, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Scaling Laws in Human Language will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-88461