Inside the Evaluation Process of the Cross-Language Evaluation Forum (CLEF): Issues of Multilingual Topic Creation and Multilingual Relevance Assessment

Proceedings of the Third International Conference on Language Resources and Evaluation (LREC 2002)

Abstract

Topic creation and relevance assessment are considered as crucial components of the evaluation process in Information Retrieval (IR). In the context of the Cross-Language Evaluation Forum (CLEF), the focus lies on evaluating multilingual functions of IR systems. Therefore, topics are generated in various languages and judging the documents delivered by the systems, requires native speakers of the participating languages who are experts in the topics’ domains. In this paper, the important issues of topic generation and relevance assessment under multilingual conditions are discussed.