Word2Vec is only a special case of Kernel Correspondence Analysis and Kernels for Natural Language Processing

We show that Correspondence Analysis (CA) is equivalent to defining the Gini-index with appropriately scaled one-hot encoding. Using this relation, we introduce nonlinear kernel extension of CA. The extended CA gives well-known analysis for categorical data (CD) and natural language processing by specializing kernels. For example, our formulation can give G-test, skip-gram with negative-sampling (SGNS), and GloVe as special cases. We introduce two kernels for natural language processing based on our formulation: a stop word (SW) kernel and a word similarity (WS) kernel. The SW kernel is a system introducing appropriate weights for SW. The WS kernel enables the use of WS test data as training data for vector space representations of words. We show that these kernels enhance accuracy when training data are insufficiently numerous.