Uncertainty Principles and Sparse Signal Representations Using Overcomplete Representations

This discussion sparse representations of signals in R. The sparsity of a signal is quantified by the number of nonzero components in its representation. Such representations of signals are useful in signal processing, lossy source coding, image processing, etc. We first speak of an uncertainty principle regarding the sparsity of any two different orthonormal basis representations of a signal S. Next, describing a signal as an overcomplete description involving a pair of orthonormal bases is considered. Because this is an overcomplete description, many possible representations exist. The hope is that the most sparse representation is a lot better than any representation using a single orthonormal basis. The uncertainty principle can be exploited to provide conditions upon when the most sparse overcomple description is unique. Next, performing the optimization search is considered, which in general is nonconvex and combinatorial. However, it is illustrated that if the most sparse representation is unique and is sufficiently sparse, it can be found using a linear programming formulation, which is considerably more computationally affordable. The notion of ’sufficiently sparse’ depends upon the pair of orthonormal bases, and in particular their mutual incoherence. We then explore typical mutual incoherence between pairs of bases, and discuss some (idealized) applications. We consider a signal S ∈ R of unit l2 energy. We are provided two different orthonormal bases A and B, described in terms of matrices where each column is one of the orthonormal vectors, i.e. A = [a1a2...aN ] and B = [b1b2...bN ]. We note that for each of the bases individually, S is uniquely