Isometric Correction for Manifold Learning

In this paper, we present a method for isometric correction of manifold learning techniques. We first present an isometric nonlinear dimension reduction method. Our proposed method overcomes the issues associated with well-known isometric embedding techniques such as ISOMAP and maximum variance unfolding (MVU), i.e., computational complexity and the geodesic convexity requirement. Based on the proposed algorithm, we derive our isometric correction method. Our approach follows an isometric solution to the problem of local tangent space alignment. We provide a derivation of a fast iterative solution. The performance of our algorithm is illustrated on both synthetic and real datasets compared to other methods. Recent advances in data acquisition and high rate information sources give rise to high volume and high dimensional data. For such data, dimension reduction provides means of visualization, compression, and feature extraction for clustering or classification. In the last decade, a variety of methods for nonlinear dimensionality reduction have been a topic of ongoing research. Common to many approaches is the geometric assumption that the data lies on a relatively low dimensional manifold embedded in a high dimensional space. The low dimensional embedding of the manifold provides means of data reduction. In manifold learning, an unorganized set of data points sampled from the manifold is used to infer about the manifold shape and geometry (Freedman 2002). Manifold learning can be viewed as either constructing the manifold from sample data points or finding an explicit map from the manifold in high-dimension to a low dimensional Euclidean space (Hoppe et al. 1994). Manifold learning and data dimension reduction have many applications, e.g., visualization, classification, and information processing. Data visualization in 2D or 3D provides further insight into the data structure, which can be used for either interpretation or data model selection. Data dimension reduction can allow for extracting meaningful features from cumbersome representations. For example, in text document classification the bag of words model offers a vector representation of the relative word frequency over a dictionary. With a large dictionary, each document can Copyright c © 2010, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. be identified with a high dimensional vector. Other aspects of data dimension reduction involve denoising or removal of redundant or irrelevant information. For example, when going after object orientation in video footage of a single object from multiple angles simultaneously, the relatively high volume information about the object shape can be discarded and only its orientation is retained. A variety of techniques for manifold learning and nonlinear data dimension reduction have been developed. ISOMAP (Tenenbaum, Silva, and Langford 2000) estimates a geodesic distance along the manifold and uses the multidimensional scaling method to embed the manifold into a low dimensional Euclidean space. Locally linear embedding (LLE) (Roweis and Saul 2000) was developed based on the linear dependence of tangent vectors on a tangent plane. Laplacian Eigenmaps (LE) (Belkin and Niyogi 2003) relies on spectral graph theory by applying an eigendecomposition to the local neighborhood graph Laplacian. Hessian Eigenmaps (Donoho and Grimes 2003) replaces the Laplacian of LE with the Hessian operator. Local Tangent Space Alignment (LTSA) (Zhang and Zha 2004) takes the approach of extracting local coordinates for each tangent plane and then constructing global coordination, which can be mapped to the local coordinates using local linear mapping. Maximum variance unfolding (MVU) (Weinberger and Saul 2006a) maximizes the data spread in the embedding space while preserving the local geometry. More about the nonlinear data dimension reduction techniques can be found in (Van Der Maaten, Postma, and Van Den Herik 2007) and (Lee and Verleysen 2007). With a handful of famous exception such as ISOMAP and MVU, none of the aforementioned techniques and other existing techniques perform an isometric embedding for a given manifold. Most of the algorithms yield a quadratic optimization objective in the form of tr ( T MT ) , where T is the global coordinates in the low dimension and M is a symmetric matrix which includes local geometry information specific to each method. The quadratic optimization problem without any constraints has degenerate solutions, e.g., T = 0 or T = [t1, t1, . . . , t1] (i.e., embed into a single point). To circumvent this problem, a set of constraints is imposed to enforce the output to (i) be centered (T 1 = 0) and (ii) have a unitary covariance (TT T = I). This set of constraints violates the isometric assumption as it distorts 2 Manifold Learning and Its Applications: Papers from the AAAI Fall Symposium (FS-10-06)