Extending Multi-Sense Word Embedding to Phrases and Sentences for Unsupervised Semantic Applications

Most unsupervised NLP models represent each word with a single point or region in semantic space, while the existing multi-sense embeddings cannot longer sequences like phrases sentences. We propose novel embedding method for text sequence (a phrase sentence) where is represented by distinct set of multi-mode codebook to capture different facets its meaning. The can be viewed as cluster centers which summarize distribution possibly co-occurring words pre-trained space. introduce an end-to-end trainable neural model that directly predicts from input during test time. Our experiments show per-sentence significantly improve performances sentence similarity and extractive summarization benchmarks. In experiments, we discover multi-facet provide interpretable representation but do not outperform single-facet baseline.