Learning Sparsely Used Overcomplete Dictionaries
نویسندگان
چکیده
We consider the problem of learning sparsely used overcomplete dictionaries, where each observation is a sparse combination of elements from an unknown overcomplete dictionary. We establish exact recovery when the dictionary elements are mutually incoherent. Our method consists of a clustering-based initialization step, which provides an approximate estimate of the true dictionary with guaranteed accuracy. This estimate is then refined via an iterative algorithm with the following alternating steps: 1) estimation of the dictionary coefficients for each observation through l1 minimization, given the dictionary estimate, and 2) estimation of the dictionary elements through least squares, given the coefficient estimates. We establish that, under a set of sufficient conditions, our method converges at a linear rate to the true dictionary as well as the true coefficients for each observation.
منابع مشابه
Exact Recovery of Sparsely Used Overcomplete Dictionaries
We consider the problem of learning overcomplete dictionaries in the context of sparse coding, where each sample selects a sparse subset of dictionary elements. Our method consists of two stages, viz., initial estimation of the dictionary, and a clean-up phase involving estimation of the coefficient matrix, and re-estimation of the dictionary. We prove that our method exactly recovers both the ...
متن کاملOvercomplete Dictionary Design by Empirical Risk Minimization
Recently, there have been a growing interest in application of sparse representation for inverse problems. Most studies concentrated in devising ways for sparsely representing a solution using a given prototype overcomplete dictionary. Very few studies have addressed the more challenging problem of construction of an optimal overcomplete dictionary, and even these were primarily devoted to the ...
متن کاملEvolution-enhanced multiscale overcomplete dictionaries learning for image denoising
In this paper, a multiscale overcomplete dictionary learning approach is proposed for image denoising by exploiting the multiscale property and sparse representation of images. The images are firstly sparsely represented by a translation invariant dictionary and then the coefficients are denoised using some learned multiscale dictionaries. Dictionaries learning can be reduced to a non-convex l0...
متن کاملCritical Points Of An Autoencoder Can Provably Recover Sparsely Used Overcomplete Dictionaries
In Dictionary Learning one is trying to recover incoherent matrices A∗ ∈ Rn×h (typically overcomplete and whose columns are assumed to be normalized) and sparse vectors x∗ ∈ R with a small support of size h for some 0 < p < 1 while being given access to observations y ∈ R where y = A∗x∗. In this work we undertake a rigorous analysis of the possibility that dictionary learning could be performed...
متن کاملEntropy of Overcomplete Kernel Dictionaries
In signal analysis and synthesis, linear approximation theory considers a linear decomposition of any given signal in a set of atoms, collected into a so-called dictionary. Relevant sparse representations are obtained by relaxing the orthogonality condition of the atoms, yielding overcomplete dictionaries with an extended number of atoms. More generally than the linear decomposition, overcomple...
متن کامل