NIST Rich Transcription 2002 Evaluation: A Preview
Proceedings of the Third International Conference on Language Resources and Evaluation (LREC 2002)
Abstract
The National Institute of Standards and Technology (NIST) has been implementing evaluations of automatic speech transcription technologies for over 15 years. NIST has helped guide progress in these technologies by: creating increasingly challenging and realistic tests, helping to provide associated linguistic resources, employing uniform metrics and analyses across systems to assess performance, and sponsoring evaluation-related technology workshops. Over time, this approach has shown great progress in the technology as the test domains have become more difficult and error rates have almost consistently decreased. In conjunction with the new DARPA Effective, Affordable, Reusable Speech (EARS) Program, NIST has begun an evaluation effort to help move the stateof- the-art to the next level in the form of a Rich Transcription (RT) evaluation program. RT is defined to be an integrated combination of speech-to-text generation (STT) and metadata (MD) annotation as applied to multiple domains such as speech from Broadcast News, telephone conversations, and meetings. The Rich Transcription 2002 (RT-02) evaluation will have been the first in an annual series of evaluations and workshops focusing on this technology.