Automatic Suggestions Help Extending Eventive Ontology: A Case Study on SynSemClass

Abstract

Despite substantial recent progress in many areas of NLP, semantic tasks remain particularly challenging. One such task is the creation (extension, or annotation) of semantic ontologies. In this work, we present a case study on the eventive SynSemClass ontology, focusing on the challenges of semantic annotation – that is extending the ontology with new lexical units and/or new concepts – both with and without automatic support. We consider two strategies for generating annotation suggestions: (i) a knowledge-driven approach based on a small, carefully curated corpus of verbal valency frames, and (ii) a corpus-driven approach using lemma-based suggestions from a large raw text collection, disregarding semantic homonymy. Our findings show that ontology annotation is inherently difficult, and that automatic annotations statistically significantly reduce this difficulty both in terms of inter-annotator agreement and when compared with gold expert annotations. We discuss the implications for semantic resource creation and extension, as well as the limits of automation in ontology annotation.

Resources

Details