Variable Word Rate N-grams

Details Variable Word Rate N-grams Variable Word Rate N-grams

2000-03-29

arxiv.org/abs/cs/0003081v1

Computer Science

Computation and Language

4 pages, 4 figures, ICASSP-2000

Scientific paper

The rate of occurrence of words is not uniform but varies from document to document. Despite this observation, parameters for conventional n-gram language models are usually derived using the assumption of a constant word rate. In this paper we investigate the use of variable word rate assumption, modelled by a Poisson distribution or a continuous mixture of Poissons. We present an approach to estimating the relative frequencies of words or n-grams taking prior information of their occurrences into account. Discounting and smoothing schemes are also considered. Using the Broadcast News task, the approach demonstrates a reduction of perplexity up to 10%.

Affiliated with

Also associated with

No associations

Rating

Variable Word Rate N-grams does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Variable Word Rate N-grams, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Variable Word Rate N-grams will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-682893