Evaluating Parsing Schemes with Entropy Indicators

Details Evaluating Parsing Schemes with Entropy Indicators Evaluating Parsing Schemes with Entropy Indicators

1997-09-22

arxiv.org/abs/cmp-lg/9709009v1

5th Meeting on Mathematics of Language, MOL5, 1997

Computer Science

Computation and Language

7 pages. LaTeX format, epsfig used, .sty file included

Scientific paper

This paper introduces an objective metric for evaluating a parsing scheme. It is based on Shannon's original work with letter sequences, which can be extended to part-of-speech tag sequences. It is shown that this regular language is an inadequate model for natural language, but a representation is used that models language slightly higher in the Chomsky hierarchy. We show how the entropy of parsed and unparsed sentences can be measured. If the entropy of the parsed sentence is lower, this indicates that some of the structure of the language has been captured. We apply this entropy indicator to support one particular parsing scheme that effects a top down segmentation. This approach could be used to decompose the parsing task into computationally more tractable subtasks. It also lends itself to the extraction of predicate/argument structure.

Affiliated with

Also associated with

No associations

Rating

Evaluating Parsing Schemes with Entropy Indicators does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Evaluating Parsing Schemes with Entropy Indicators, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Evaluating Parsing Schemes with Entropy Indicators will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-504393