Fast Subspace Approximation via Greedy Least-Squares

In this note, we develop fast and deterministic dimensionality reduction techniques for a family of subspace approximation problems. Let P ⊂ R be a given set of M points. The techniques developed herein find an O(n logM)-dimensional subspace that is guaranteed to always contain a near-best fit n-dimensional hyperplane H for P with respect to the cumulative projection error (∑ x∈P ‖x−ΠHx‖ p 2 )1/p , for any chosen p > 2. The deterministic algorithm runs in Õ ( MN ) -time, and can be randomized to run in only Õ (MNn)-time while maintaining its error guarantees with high probability. In the case p = ∞ the dimensionality reduction techniques can be combined with efficient algorithms for computing the John ellipsoid of a data set in order to produce an n-dimensional subspace whose maximum `2-distance to any point in the convex hull of P is minimized. The resulting algorithm remains Õ (MNn)-time. In addition, the dimensionality reduction techniques developed herein can also be combined with other existing subspace approximation algorithms for 2 < p ≤ ∞ – including more accurate algorithms based on convex programming relaxations – in order to reduce their runtimes.