Fuzzy V-Measure - An Evaluation Method for Cluster Analyses of Ambiguous Data

Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC 2014)

Abstract

This paper discusses an extension of the V-measure (Rosenberg and Hirschberg, 2007), an entropy-based cluster evaluation metric. While the original work focused on evaluating hard clusterings, we introduce the Fuzzy V-measure which can be used on data that is inherently ambiguous. We perform multiple analyses varying the sizes and ambiguity rates and show that while entropy-based measures in general tend to suffer when ambiguity increases, a measure with desirable properties can be derived from these in a straightforward manner.