Computer Science – Computation and Language
Scientific paper
2008-01-24
Proceedings of the 3rd Language & Technology Conference: Human Language Technologies as a Challenge for Computer Science and L
Computer Science
Computation and Language
Scientific paper
Robustness in a parser refers to an ability to deal with exceptional phenomena. A parser is robust if it deals with phenomena outside its normal range of inputs. This paper reports on a series of robustness evaluations of state-of-the-art parsers in which we concentrated on one aspect of robustness: its ability to parse sentences containing misspelled words. We propose two measures for robustness evaluation based on a comparison of a parser's output for grammatical input sentences and their noisy counterparts. In this paper, we use these measures to compare the overall robustness of the four evaluated parsers, and we present an analysis of the decline in parser performance with increasing error levels. Our results indicate that performance typically declines tens of percentage units when parsers are presented with texts containing misspellings. When it was tested on our purpose-built test set of 443 sentences, the best parser in the experiment (C&C parser) was able to return exactly the same parse tree for the grammatical and ungrammatical sentences for 60.8%, 34.0% and 14.9% of the sentences with one, two or three misspelled words respectively.
No associations
LandOfFree
Robustness Evaluation of Two CCG, a PCFG and a Link Grammar Parsers does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Robustness Evaluation of Two CCG, a PCFG and a Link Grammar Parsers, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Robustness Evaluation of Two CCG, a PCFG and a Link Grammar Parsers will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-283591