A variation of the differential steepest descent algorithm, here called the dithered linear search (DLS), is examined and applied to analog filter adaptation. The DLS algorithm is a gradient descent optimizer with a straightforward and robust hardware implementation. Gradient estimates are obtained by applying independent additive dither to all of the filter’s parameters simultaneously and corr...

NOISY GRADIENT DESCENT BIT FLIP DECODING OF LOW DENSITY PARITY CHECK CODES: ALGORITHM AND IMPLEMENTATION

With the advent of high-throughput technologies, l1 regularized learning algorithms have attracted much attention recently. Dozens of algorithms have been proposed for fast implementation, using various advanced optimization techniques. In this paper, we demonstrate that l1 regularized learning problems can be easily solved by using gradient-descent techniques. The basic idea is to transform a ...

Prediction of traffic is very crucial for its management. Because of human involvement in the generation of this phenomenon, traffic signal is normally accompanied by noise and high levels of non-stationarity. Therefore, traffic signal prediction as one of the important subjects of study has attracted researchers’ interests. In this study, a combinatorial approach is proposed for traffic signal...

In this paper, we proposed a new hybrid conjugate gradient algorithm for solving unconstrained optimization problems as convex combination of the Dai-Yuan algorithm, conjugate-descent and Hestenes-Stiefel algorithm. This is globally convergent satisfies sufficient descent condition by using strong Wolfe conditions. The numerical results show that nonlinear efficient robust.

The aim of this paper is to detect bank credit cards related frauds. The large amount of data and their similarity lead to a time consuming and low accurate separation of healthy and unhealthy samples behavior, by using traditional classifications. Therefore in this study, the Adaptive Neuro-Fuzzy Inference System (ANFIS) is used in order to reach a more efficient and accurate algorithm. By com...

this paper intends to offer a new iterative method based on articial neural networks for finding solution of a fuzzy equations system. our proposed fuzzied neural network is a ve-layer feedback neural network that corresponding connection weights to output layer are fuzzy numbers. this architecture of articial neural networks, can get a real input vector and calculates its corresponding fu...

We evaluate natural gradient, an algorithm originally proposed in Amari (1997), for learning deep models. The contributions of this paper are as follows. We show the connection between natural gradient and three other recently proposed methods: Hessian-Free (Martens, 2010), Krylov Subspace Descent (Vinyals and Povey, 2012) and TONGA (Le Roux et al., 2008). We empirically evaluate the robustness...

This paper presents an axiomatic approach for constructing radial basis function (RBF) neural networks. This approach results in a broad variety of admissible RBF models, including those employing Gaussian RBF's. The form of the RBF's is determined by a generator function. New RBF models can be developed according to the proposed approach by selecting generator functions other than exponential ...

We consider the problem of constructing an aggregated estimator from a finite class of base functions which approximately minimizes a convex risk functional under the l1 constraint. For this purpose, we propose a stochastic procedure, the mirror descent, which performs gradient descent in the dual space. The generated estimates are additionally averaged in a recursive fashion with specific weig...