Imposing sparse + group-sparse superposition structures in high-dimensional parameter estimation is known to provide flexible regularization that is more realistic for many real-world problems. For example, such a superposition enables partially-shared support sets in multi-task learning, thereby striking the right balance between parameter overlap across tasks and task specificity. Existing th...

Deconvolution refers to the problem of estimating the unknown input to an LTI system when the output signal and system response are known. In practice, the available output signal is also noisy. For some systems, the deconvolution problem is quite straight forward; however, for systems that are non-invertible or nearly non-invertible (e.g. narrow-band or with frequency response nulls), the prob...

There exist many storage formats for the in-memory representation of sparse matrices. Choosing the format that yields the quickest processing of any given sparse matrix requires considering the exact non-zero structure of the matrix, as well as the current execution environment. Each of these factors can change at runtime. The matrix structure can vary as computation progresses, while the envir...

The Recursive Sparse Blocks (RSB) is a sparse matrix layout designed for coarse grained parallelism and reduced cache misses when operating with matrices, which are larger than a computer’s cache. By laying out the matrix in sparse, non overlapping blocks, we allow for the shared memory parallel execution of transposed SParse Matrix-Vector multiply (SpMV ), with higher efficiency than the tradi...

consider the case where x is has exactly k + 1 nonzero entries. Then, Errk(x) = Err 2 k(x) = x (k+1), where x(k+1) represents the smallest of the k + 1 nonzero entries in x. Thus, the formula above implies that for such x, the LP finds an x∗ that is better than the best k-sparse approximation, so clearly x∗ cannot be k-sparse. In practice, it is often not important that x∗ be k-sparse. For exam...

Sudocodes are a new scheme for lossless compressive sampling and reconstruction of sparse signals. Consider a sparse signal x ∈ R containing only K N non-zero values. Sudo-encoding computes the codeword y ∈ R via the linear matrix-vector multiplication y = Φx, with K < M N . We propose a non-adaptive construction of a sparse Φ comprising only the values 0 and 1; hence the computation of y invol...

The randomized version of the Kaczmarz method for the solution of linear systems is known to converge linearly in expectation. In this work we extend this result and show that the recently proposed Randomized Sparse Kaczmarz method for recovery of sparse solutions, as well as many variants, also converges linearly in expectation. The result is achieved in the framework of split feasibility prob...

In this work we provide a convergence analysis for the quasi-optimal version of the Stochastic Sparse Grid Collocation method we had presented in our previous work “On the optimal polynomial approximation of Stochastic PDEs by Galerkin and Collocation methods” [6]. Here the construction of a sparse grid is recast into a knapsack problem: a profit is assigned to each hierarchical surplus and onl...