Usability Evaluation of Spoken Dialogue Systems

Abstract

This paper concerns the methodologies currently applied to evaluation of spoken and multi modal dialogue systems. Usability is concerned with the effectiveness, efficiency and satisfaction by which users achieves their goals when using a system. The methods developed and applied by the speech community over the past decade to measure the usability of Spoken Dialogue Systems (SDS) are discussed and critically reviewed. The paper starts by giving a general review and discussion of the current issues and problems of establishing SDS usability. This is supported by the presentation and analysis of a case, where it is shown that e.g. learnability can be derived from simple performance measures, such as duration and turn-taking. Results of applying the PARADISE method are presented and discussed.

Resources

Details