Summary of the paper

Title Automatic Evaluation Measures for Statistical Machine Translation System Optimization
Authors Arne Mauser, Sasa Hasan and Hermann Ney
Abstract Evaluation of machine translation (MT) output is a challenging task. In most cases, there is no single correct translation. In the extreme case, two translations of the same input can have completely different words and sentence structure while still both being perfectly valid. Large projects and competitions for MT research raised the need for reliable and efficient evaluation of MT systems. For the funding side, the obvious motivation is to measure performance and progress of research. This often results in a specific measure or metric taken as primarily evaluation criterion. Do improvements in one measure really lead to improved MT performance? How does a gain in one evaluation metric affect other measures? This paper is going to answer these questions by a number of experiments.
Language
Topics Machine Translation, SpeechToSpeech Translation, Evaluation methodologies, Statistical methods
Full paper Automatic Evaluation Measures for Statistical Machine Translation System Optimization
Slides -
Bibtex @InProceedings{MAUSER08.785,
  author = {Arne Mauser, Sasa Hasan and Hermann Ney},
  title = {Automatic Evaluation Measures for Statistical Machine Translation System Optimization},
  booktitle = {Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)},
  year = {2008},
  month = {may},
  date = {28-30},
  address = {Marrakech, Morocco},
  editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Daniel Tapias},
  publisher = {European Language Resources Association (ELRA)},
  isbn = {2-9517408-4-0},
  note = {http://www.lrec-conf.org/proceedings/lrec2008/},
  language = {english}
  }

Powered by ELDA © 2008 ELDA/ELRA