LREC 2000 2nd International Conference on Language Resources & Evaluation
 

Previous Paper   Next Paper

Title Principled Hidden Tagset Design for Tiered Tagging of Hungarian
Authors Tufiş Dan (RACAI-Romanian Academy 13, “13 Septembrie”, Ro-74311, Bucharest 5, Romania, email:tufis@valhalla.racai.ro)
Dienes Péter (Research Institute for Linguistics, Hungarian Academy of Sciences, Budapest, dienes@nytud.hu)
Oravecz Csaba (Research Institute for Linguistics, Hungarian Academy of Sciences, Budapest, oravecz@nytud.hu)
Váradi Tamás (Linguistics Institute, Hungarian Academy of Sciences, H-1014 Budapest Színház u 5-9, varadi@nytud.hu)
Keywords Corpus Annotation, Tagset Design, Tagset Reduction, Tiered Tagging
Session Session WO18 - Morphology in Lexical and Textual Resources
Full Paper 249.ps, 249.pdf
Abstract For highly inflectional languages, the number of morpho-syntactic descriptions (MSD), required to descriptionally cover the content of a word-form lexicon, tends to rise quite rapidly, approaching a thousand or even more set of distinct codes. For the purpose of automatic disambiguation of arbitrary written texts, using such large tagsets would raise very many problems, starting from implementation issues of a tagger to work with such a large tagsets to the more theory-based difficulty of sparseness of training data. Tiered tagging is one way to alleviate this problem by reformulating it in the following way: starting from a large set of MSDs, design a reduced tagset, Ctag-set, manageable for the current tagging technology. We describe the details of the reduced tagset design for Hungarian, where the MSD-set cardinality is several thousand. This means that designing a manageable C-tagset calls for severe reduction in the number of the MSD features, a process that requires careful evaluation of the features.