Back to Main Conference 2004
LREC 2004main
Agreement in Human Factoid Annotation for Summarization Evaluation
Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC 2004)
Abstract
Factoid analysis was introduced by vanHalteren and Teufel (2003) as an objective, yet semantics-oriented way of measuring overlap of information rather than surface strings in summaries. In this paper, we report on annotation experiments with two sets of summaries, and on a factoid-pairing program which finds correlations between factoids semi-automatically.