Experiments with LSA Scoring: Optimal Rank and Basis

Latent Semantic Analysis (LSA) is one of the main variants of vector space methods for information retrieval, and continues to be an active area of research, both in the theory of how LSA works and in the practical applications of the method. Alternatives to the Singular Value Decomposition (SVD) have been explored that o er improvements in storage, speed, updating or other advantages, especially for large datasets. Alternatives to constructing the term-document matrix, such as term-weighting schemes have also been explored. At the core of LSA is the scoring of a query against a canonical set of documents, using the inner product of their vector representations. Virtually all LSA researchers have assumed a particular scoring function, and alternatives have not been well investigated. In this paper I de ne a family of scoring functions parameterized by a "weighting exponent" p, which weights the score by , where is the diagonal matrix of SVD eigenvalues. I present empirical ndings on how retrieval accuracy varies as a function of p and the SVD approximation rank k, and nd that often the standard scoring function, corresponding to p = 0, is not optimal. In Section 2, I review the Singular Value Decomposition, and de ne LSA scoring functions as a choice of basis for the vector space de ned by the document set. In Section 3, I test these scoring functions on standard test datasets, for di erent values of the SVD approximation rank k. In Section 4, I present results from a similar experiment on matching question with answers, within the context of the Frequently Asked Question Organizer (FAQO), a prototype system for technical support that I developed. In Section 5, I summarize related work, and discuss the signi cance of these results.