Evaluation of Multi-party Virtual Reality Dialogue Interaction

Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC 2004)

Abstract

We describe a dialogue evaluation plan for a multi-character virtual reality training simulation. A multi-component evaluation plan is presented, including user satisfaction, intended task completion, recognition rate, and a new annotation scheme for appropriateness. Preliminary results for formative tests are also presented.