Topic modeling and word embedding are two important techniques for deriving latent semantics from data. General-purpose topic models typically work in coarse granularity by capturing word co-occurrence at the document/sentence level. In contrast, word embedding models usually work in fine granularity by modeling word co-occurrence within small sliding windows. With the aim of deriving latent se...

This paper presents a detailed survey of word co-occurrence measures used in natural language processing. Word co-occurrence information is vital for accurate computational text treatment, it is important to distinguish words which can combine freely with other words from other words whose preferences to generate phrases are restricted. The latter words together with their typical co-occurring ...

We address the issue of data sparseness problem in language model (LM). Using class LM is one way to avoid this problem. In class LM, infrequent words are supported by more frequent words in the same class. This paper investigates a class LM based on LSA. A word-document matrix is usually used to represent a corpus in LSA framework. However, this matrix ignores word order in the sentence. We pr...

A data sparseness problem for modeling a language often occurs in many language models (LMs). This problem is caused by the insufficiency of training data, which in turn, makes the infrequent words have unreliable probability. Mapping from words into classes gives the infrequent words more confident probability, because they can rely on other more frequent words in the same class. In this resea...

This paper is focused on one aspect of SOPMI, an unsupervised approach to sentiment vocabulary acquisition proposed by Turney (Turney and Littman, 2003). The method, originally applied and evaluated for English, is often used in bootstrapping sentiment lexicons for European languages where no such resources typically exist. In general, SO-PMI values are computed from word co-occurrence frequenc...

Co-occurrence analysis has been used to determine related words or terms in many NLP-related applications such as query expansion in Information Retrieval (IR). However, related words are usually determined with respect to a single word, without relevant information for its application context. For example, the word “programming” may be considered to be strongly related to “Java”, and applied i...

Word Sense Induction (WSI) is the task of identifying the different senses (uses) of a target word in a given text. Traditional graph-based approaches create and then cluster a graph, in which each vertex corresponds to a word that co-occurs with the target word, and edges between vertices are weighted based on the co-occurrence frequency of their associated words. In contrast, in our approach ...

A triangulation strategy, employing a number of network analysis techniques, was implemented in the study of a single social network of biomedical scientists specializing in lipid metabolism research. Here we present the results of co-word analysis of grants awarded to these scientists by the National Institutes of Health, network analysis (NEGOPY) and factor analysis of the scientists’ respons...

Non-negative Matrix Factorization (NMF) and its variants have been successfully used for clustering text documents. However, NMF approaches like other models do not explicitly account the contextual dependencies between words. To remedy this limitation, we draw inspiration from neural word embedding posit that words frequently co-occur within same context (e.g., sentence or document) are likely...