Title

Multimodal Systems, Resources and Evaluation

Authors

Mark T. Maybury (Information Technology Division The MITRE Corporation 202 Burlington Road Bedford, MA 01730, USA)

Session

Keynote

Abstract

This paper considers multimodal systems, resources, and evaluation. We first motivate the value of multimodal information access with a vision of multimodal question answering and an example of content based access to broadcast news video. We next describe intelligent multimodal interfaces, define terminology, and summarize a range of applications, required corpora, and associated media. We then introduce a jointly created roadmap for multimodality and show an example of an open source multimodal spoken dialogue toolkit. We next describe requirements for and an abstract architecture of multimodal systems. We conclude discussing multimodal collaboration, multimodal instrumentation, and multilevel evaluation.

Keywords

Multimodal systems

Full Paper

336.pdf