Extraction of Collocations from a Text Corpus: A Fuzzy Measure

Automatic extraction of collocations from a corpus is a well-known problem in the field of natural language processing. It is typically carried out by employing some kind of a statistical measure that indicates whether or not two words occur together more often than by chance. A fuzzy set theoretic approach for extracting collocations from a text collection is described in this article. This approach proposed a fuzzy bi-gram index to find the bi-grams from a collection. Collocations of higher length i.e., ngrams (n>2) are then obtained using the fuzzy bi-gram index where the extracted collocations of lower lengths are treated as individual words. The performance of the proposed methods is found to be quite promising and it is better than that of other widely used methods we considered. Keywords—Collocation extraction, Fuzzy sets, Natural language processing, Corpus statistics, GENIA corpus.