Back to Main Conference 2002
LREC 2002main
End-to-End Evaluation of Multimodal Dialogue Systems – can we Transfer Established Methods?
Proceedings of the Third International Conference on Language Resources and Evaluation (LREC 2002)
Abstract
The goal of this paper is to define a methodology for the end-to-end evaluation of the multimodal dialogue system SmartKom along the lines of the DARPA guidelines for spoken dialogue systems. The methodology consists of an extended framework for the evaluation of a multimodal dialogue system, evaluation metrics for its various components,and an approach to compare the user satisfaction with the system's technical performance.