Inducing Features of Random Fields

Details Inducing Features of Random Fields Inducing Features of Random Fields

1995-06-13

arxiv.org/abs/cmp-lg/9506014v1

Computer Science

Computation and Language

34 pages, compressed postscript

Scientific paper

We present a technique for constructing random fields from a set of training samples. The learning paradigm builds increasingly complex fields by allowing potential functions, or features, that are supported by increasingly large subgraphs. Each feature has a weight that is trained by minimizing the Kullback-Leibler divergence between the model and the empirical distribution of the training data. A greedy algorithm determines how features are incrementally added to the field and an iterative scaling algorithm is used to estimate the optimal values of the weights. The statistical modeling techniques introduced in this paper differ from those common to much of the natural language processing literature since there is no probabilistic finite state or push-down automaton on which the model is built. Our approach also differs from the techniques common to the computer vision literature in that the underlying random fields are non-Markovian and have a large number of parameters that must be estimated. Relations to other learning approaches including decision trees and Boltzmann machines are given. As a demonstration of the method, we describe its application to the problem of automatic word classification in natural language processing. Key words: random field, Kullback-Leibler divergence, iterative scaling, divergence geometry, maximum entropy, EM algorithm, statistical learning, clustering, word morphology, natural language processing

Affiliated with

Also associated with

No associations

Rating

Inducing Features of Random Fields does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Inducing Features of Random Fields, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Inducing Features of Random Fields will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-665708