A Protocol for Evaluating Analyzers of Syntax (PEAS)
Proceedings of the Third International Conference on Language Resources and Evaluation (LREC 2002)
Abstract
Providing a comparative framework for parsers is a task that has already been tried in the past, e.g. (Abeill´e, 1991), (Atwell and Sutcliffe, 1997), (Black et al., 1991), and studied in the literature (Black, 1993), (Black, 1994), (Carroll et al., 1998), (Gaizauskas et al., 1998), (WEPS-98, ), (Mengel and Lezius, 2000), but mainly for English. In this paper, we present PEAS: a Protocol for Evaluating Analyzers of Syntax (in French: Protocole d’Evaluation pour les Analyseurs Syntaxiques), based on an ongoing experiment at LIMSI which aims at developing and testing a generic quantitative black-box evaluation protocol for parsers of French. Two fully operational parsers will be used to test the evaluation protocol; they are: the parser (Giguet and Vergne, 1997) developed at GREYC (Caen University) and the latest version of the parser developed at Rank Xerox Research Center in Grenoble (Ait-Mokhtar and Chanod, 1997)