ColLex.en: Automatically Generating and Evaluating a Full-form Lexicon for English

Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC 2014)

Abstract

The paper describes a procedure for the automatic generation of a large full-form lexicon of English. We put emphasis on two statistical methods to lexicon extension and adjustment: in terms of a letter-based HMM and in terms of a detector of spelling variants and misspellings. The resulting resource, \collexen, is evaluated with respect to two tasks: text categorization and lexical coverage by example of the SUSANNE corpus and the \openanc.