Computer Science – Computation and Language
Scientific paper
1996-12-31
Computer Science
Computation and Language
32 pages, texinfo format
Scientific paper
The Maximum Entropy Modeling Toolkit supports parameter estimation and prediction for statistical language models in the maximum entropy framework. The maximum entropy framework provides a constructive method for obtaining the unique conditional distribution p*(y|x) that satisfies a set of linear constraints and maximizes the conditional entropy H(p|f) with respect to the empirical distribution f(x). The maximum entropy distribution p*(y|x) also has a unique parametric representation in the class of exponential models, as m(y|x) = r(y|x)/Z(x) where the numerator m(y|x) = prod_i alpha_i^g_i(x,y) is a product of exponential weights, with alpha_i = exp(lambda_i), and the denominator Z(x) = sum_y r(y|x) is required to satisfy the axioms of probability. This manual explains how to build maximum entropy models for discrete domains with the Maximum Entropy Modeling Toolkit (MEMT). First we summarize the steps necessary to implement a language model using the toolkit. Next we discuss the executables provided by the toolkit and explain the file formats required by the toolkit. Finally, we review the maximum entropy framework and apply it to the problem of statistical language modeling. Keywords: statistical language models, maximum entropy, exponential models, improved iterative scaling, Markov models, triggers.
No associations
LandOfFree
Maximum Entropy Modeling Toolkit does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Maximum Entropy Modeling Toolkit, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Maximum Entropy Modeling Toolkit will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-304105