Parsing Ungrammatical Input: an Evaluation Procedure

Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC 2004)

Abstract

This paper presents a procedure for evaluating a parser's ability to produce an accurate parse for an ungrammatical sentence. It is based on the existence of a corpus of ungrammatical sentences, and a parallel corpus containing corrected, and hence grammatical, versions of the sentences in the first corpus. This procedure is applied to a wide-coverage probabilistic parser (Charniak, 2002), and the performance of this parser with respect to ungrammatical input is analysed.