Title

Multi-lingual Evaluation of a Natural Language Generation System

Author(s)

Athanasios Karasimos (1), Amy Isard (2)

(1) Theoretical and Applied Linguistics, University of Edinburgh; (2) School of Informatics, University of Edinburgh

Session

O21-EW

Abstract

This paper describes a user evaluation of the text output from the M-PIRO (Multilingual Personalised Information Objects) system, which dynamically generates descriptions of exhibits for a virtual museum. We show that subjects performed significantly better in a factual recall test when the descriptions included more sophisticated text structuring modules. The subjects also judged the structured texts to be more interesting and readable, and felt that they had learned more from them.

Keyword(s)

evaluation, natural language generation, aggregation, comparison

Language(s) English, Greek
Full Paper

134.pdf