| Title |
Inside the Evaluation Process of the Cross-Language Evaluation Forum (CLEF): Issues of Multilingual Topic Creation and Multilingual Relevance Assessment |
| Authors |
Michael Kluck (Social Science Information Centre (IZ) Lennéstr. 30, 53113 Bonn, Germany) Christa Womser-Hacker (University of Hildesheim, Institute for Applied Linguistics, Information Science Marienburger Platz 22, 31141 Hildesheim, Germany) |
| Session |
EP1: Evaluation |
| Abstract |
Topic creation and relevance assessment are considered as crucial components of the evaluation process in Information Retrieval (IR). In the context of the Cross-Language Evaluation Forum (CLEF), the focus lies on evaluating multilingual functions of IR systems. Therefore, topics are generated in various languages and judging the documents delivered by the systems, requires native speakers of the participating languages who are experts in the topics’ domains. In this paper, the important issues of topic generation and relevance assessment under multilingual conditions are discussed. |
| Keywords |
Multilingual |
| Full Paper |