For high-dimensional sparse parameter estimation problems, Log-Sum Penalty (LSP) regularization effectively reduces the sampling sizes in practice. However, it still lacks theoretical analysis to support the experience from previous empirical study. The analysis of this article shows that, like `0-regularization, O(s) sampling size is enough for proper LSP, where s is the non-zero components of...

We propose two affine projection algorithms (APA) with variable regularization parameter. The proposed algorithms dynamically update the regularization parameter that is fixed in the conventional regularized APA (R-APA) using a gradient descent based approach. By introducing the normalized gradient, the proposed algorithms give birth to an efficient and a robust update scheme for the regulariza...

This paper proposes a novel regularization approach for Extreme Learning Machines. Regularization is performed using a priori spacial information expressed by an affinity matrix. We show that the use of this type of a priori information is similar to perform Tikhonov regularization. Furthermore, if a parameter free affinity matrix is used, like the cosine similarity matrix, regularization is pe...

In the solution of ill-posed problems by means of regularization methods, a crucial issue is the computation of the regularization parameter. In this work, we focus on the Truncated Singular Value Decomposition (TSVD) and Tikhonov method, and we define a method for computing the regularization parameter based on the behavior of Fourier coefficients. We compute a safe index for truncating the TS...

Generalized Cross Validation (GCV) is a popular approach to determining the regularization parameter in Tikhonov regularization. The regularization parameter is chosen by minimizing an expression, which is easy to evaluate for small-scale problems, but prohibitively expensive to compute for large-scale ones. This paper describes a novel method, based on Gauss-type quadrature, for determining up...

In this paper we study automatic regularization techniques for the fusion of automatic speaker recognition systems. Parameter regularization could dramatically reduce the fusion training time. In addition, there will not be any need for splitting the development set into different folds for crossvalidation. We utilize majorization-minimization approach to automatic ridge regression learning and...

In image processing, it is often desirable to remove noise, smooth or sharpen image features, or to otherwise enhance the image. Total Variation (TV) based regularization is a model case of geometry-driven diiusion for image processing. In our papers 14] and 15], we analyze the precise eeects of TV based regularization by analytically nding exact solutions to the TV regularization problem. In t...

A simple proof of the convergence of the variational regularization, with the regularization parameter, chosen by the discrepancy principle, is given for linear operators under suitable assumptions. It is shown that the discrepancy principle, in general, does not yield uniform with respect to the data convergence. An a priori choice of the regularization parameter is proposed and justified for ...

We discuss the solution of numerically ill-posed overdetermined systems of equations using Tikhonov a-priori-based regularization. When the noise distribution on the measured data is available to appropriately weight the fidelity term, and the regularization is assumed to be weighted by inverse covariance information on the model parameters, the underlying cost functional becomes a random varia...

A long-standing problem in classification is the determination of the regularization parameter. Nearly every classification algorithm uses a parameter (or set of parameters) to control classifier complexity. Crossvalidation on the training set is usually done to determine the regularization parameter(s). [1] proved a leave-one-out cross-validation (LOOCV) bound for a class of kernel classifiers...