A Dataset for Assessing Machine Translation Evaluation Metrics

Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC 2010)

Abstract

We describe a dataset containing 16,000 translations produced by four machine translation systems and manually annotated for quality by professional translators. This dataset can be used in a range of tasks assessing machine translation evaluation metrics, from basic correlation analysis to training and test of machine learning-based metrics. By providing a standard dataset for such tasks, we hope to encourage the development of better MT evaluation metrics.