Computer Science – Computation and Language
Scientific paper
1994-12-06
Computer Science
Computation and Language
27 pages, latex, comments welcome
Scientific paper
Statistical language models frequently suffer from a lack of training data. This problem can be alleviated by clustering, because it reduces the number of free parameters that need to be trained. However, clustered models have the following drawback: if there is ``enough'' data to train an unclustered model, then the clustered variant may perform worse. On currently used language modeling corpora, e.g. the Wall Street Journal corpus, how do the performances of a clustered and an unclustered model compare? While trying to address this question, we develop the following two ideas. First, to get a clustering algorithm with potentially high performance, an existing algorithm is extended to deal with higher order N-grams. Second, to make it possible to cluster large amounts of training data more efficiently, a heuristic to speed up the algorithm is presented. The resulting clustering algorithm can be used to cluster trigrams on the Wall Street Journal corpus and the language models it produces can compete with existing back-off models. Especially when there is only little training data available, the clustered models clearly outperform the back-off models.
No associations
LandOfFree
An Extended Clustering Algorithm for Statistical Language Models does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with An Extended Clustering Algorithm for Statistical Language Models, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and An Extended Clustering Algorithm for Statistical Language Models will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-180085