A Decomposition Algorithm for Sparse Generalized Eigenvalue Problem

Sparse generalized eigenvalue problem arises in a number of standard and modern statistical learning models, including sparse principal component analysis, sparse Fisher discriminant analysis and sparse canonical correlation analysis. However, this problem is difficult to solve since it is NP-hard. In this paper, we consider a new decomposition method to tackle this problem. Specifically, we use random or/and swapping strategies to find a working set and perform global combinatorial search over the small subset of variables. We consider a bisection search method and a coordinate descent method for solving the quadratic fractional programming subproblem. In addition, we provide theoretical analysis for the proposed decomposition algorithm. Extensive experiments on both real-world and synthetic data sets have shown that the proposed method achieves stateof-the-art performance in terms of accuracy.