Multi-lingual Evaluation of a Natural Language Generation System

Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC 2004)

Abstract

This paper describes a user evaluation of the text output from the M-PIRO (Multilingual Personalised Information Objects) system, which dynamically generates descriptions of exhibits for a virtual museum. We show that subjects performed significantly better in a factual recall test when the descriptions included more sophisticated text structuring modules. The subjects also judged the structured texts to be more interesting and readable, and felt that they had learned more from them.