Textual Distraction as a Basis for Evaluating Automatic Summarisers

Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC 2004)

Abstract

Our summarisation tool, SEAGULL (Summary Extraction Algorithm Generated Using Lexical Links), is a sentence extractor which exploits the patterns of lexical repetition across a text and creates abridgements which express non-trivially the conceptual content and development of topic. In this paper, we report on a test devised to assess its performance against other summarisers. This involves the introduction of progressive batches of unrelated sentences into a source text. Targeted distraction reveals the relative degrees of robustness in summariser performance. The tests show that our system functions best.