Topic Modeling via Nonnegative Matrix Factorization on Probability Simplex

One important goal of document modeling is to extract a set of informative topics from a text corpus and produce a reduced representation of each document. In this paper, we propose a novel algorithm for this task based on nonnegative matrix factorization on a probability simplex. We further extend our algorithm by removing global and generic information to produce more diverse and specific topics. In contrast to other matrix factorization methods, such as latent semantic indexing by singular value decomposition, our model has a solid statistical foundation and is based on a generative model for text corpus. In contrast to purely probabilistic approach, such as probabilistic latent semantic indexing (pLSI) solved by Expectation-Maximization, our method is based on efficient block coordinate decent optimization. Experiments demonstrate that the new method generates more meaningful and diverse topics compared with pLSI and LDA with faster convergence behavior.