For Most Large Underdetermined Systems of Equations, the Minimal `-norm Near-Solution Approximates the Sparsest Near-Solution

We consider inexact linear equations y ≈ Φα where y is a given vector in R, Φ is a given n by m matrix, and we wish to find an α0, which is sparse and gives an approximate solution, obeying ‖y − Φα0, ‖2 ≤ . In general this requires combinatorial optimization and so is considered intractable. On the other hand, the ` minimization problem min ‖α‖1 subject to ‖y − Φα‖2 ≤ , is convex, and is considered tractable. We show that for most Φ the solution α̂1, = α̂1, (y,Φ) of this problem is quite generally a good approximation for α̂0, . We suppose that the columns of Φ are normalized to unit ` norm 1 and we place uniform measure on such Φ. We study the underdetermined case where m ∼ An, A > 1 and prove the existence of ρ = ρ(A) and C > 0 so that for large n, and for all Φ’s except a negligible fraction, the following approximate sparse solution property of Φ holds: For every y having an approximation ‖y − Φα0‖2 ≤ by a coefficient vector α0 ∈ R with fewer than ρ · n nonzeros, we have ‖α̂1, − α0‖2 ≤ C · . This has two implications. First: for most Φ, whenever the combinatorial optimization result α0, would be very sparse, α̂1, is a good approximation to α0, . Second: suppose we are given noisy data obeying y = Φα0 + z where the unknown α0 is known to be sparse and the noise ‖z‖2 ≤ . For most Φ, noise-tolerant `-minimization will stably recover α0 from y in the presence of noise z. We study also the barely-determined casem = n and reach parallel conclusions by slightly different arguments. The techniques include the use of almost-spherical sections in Banach space theory and concentration of measure for eigenvalues of random matrices.