| Title |
Galaxy-II as an Architecture for Spoken Dialogue Evaluation |
| Authors |
Polifroni Joseph (Spoken Language Systems Group, MIT Laboratory for Computer Science, 545 Technology Square, Cambridge, MA 02139 USA, joe@lcs.mit.edu)
Seneff Stephanie (Spoken Language Systems Group, MIT Laboratory for Computer Science, 545 Technology Square, Cambridge, MA 02139 USA, seneff@lcs.mit.edu) |
| Keywords |
Evaluation, Spoken Dialogue Architectures, Spoken Dialogue Dystems |
| Session |
Session SO5 - Evaluation of Dialogue |
| Full Paper |
116.ps, 116.pdf |
| Abstract |
The GALAXY-II architecture, comprised of a centralized hub mediating the interaction among a suite of human language technology servers, provides both a useful tool for implementing systems and also a streamlined way of configuring the evaluation of these systems. In this paper, we discuss our ongoing efforts in evaluation of spoken dialogue systems, with particular attention to the way in which the architecture facilitates the development of a variety of evaluation configurations. We furthermore propose two new metrics for automatic evaluation of the discourse and dialogue components of a spoken dialogue system, which we call “user frustration” and “information bit rate.” |
