A Corpus-based Approach to the Interpretation of Unknown Words with an Application to German
Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC 2006)
Abstract
Usually a high portion of the different word forms in a corpusreceive no reading by the lexical and/or morphological analysis.These unknown words constitute a huge problem for NLP analysis tasks likePOS-tagging or syntactic parsing. We present a parameterizable (in principle language-independent) corpus-basedapproach for the interpretation of unknown words that only needs a tokenizedcorpus and can be used in both offline and online applications. In combination with a few linguistic (language-dependent) rules unknown verbs, adjectives, nouns, multiword units etc. are identified.Depending on the recognized word class(es), more detailed morphosyntactic and semantic information is additionally identified in opposite to the majority ofother unknown word guessing methods,which only uses a very narrow decision window to assign an unknown wordits correct reading respective Part-of-Speech tag in a given text. We tested our approach by experiments with German data and received very promising results.