IDR(s) is one of the most efficient methods for solving large sparse nonsymmetric linear systems of equations. We present two useful extensions of IDR(s), namely a flexible variant and a multi-shift variant. The algorithms exploit the underlying Hessenberg decomposition computed by IDR(s) to generate basis vectors for the Krylov subspace. The approximate solution vectors are computed using a Qu...

Preconditioning  Krylov Subspace Methods are commonly used for solving linear system  Standard implementations are communication-bound due to required SpMV and orthogonalization in every iteration  Solution: rearrange algorithms to perform s iterations at a time without communicating (s-step methods)  SpMV in each iteration is replaced with a call to the Matrix Powers Kernel, which performs...

Consider a system of linear algebraic equations Ax = b where A is an n by n real matrix and b a real vector of length n. Unlike in the linear iterative methods based on the idea of splitting of A, the Krylov subspace methods, which are used in computational kernels of various optimization techniques, look for some optimal approximate solution xn in the subspaces Kn(A, b) = span{b, Ab, . . . , A...

Krylov subspace methods often exhibit superlinear convergence. We present a general analytic model which describes this superlinear convergence, when it occurs. We take an invariant subspace approach, so that our results apply also to inexact methods, and to non-diagonalizable matrices. Thus, we provide a unified treatment of the superlinear convergence of GMRES, Conjugate Gradients, block vers...

We consider deflation and augmentation techniques for accelerating the convergence of Krylov subspace methods for the solution of nonsingular linear algebraic systems. Despite some formal similarity, the two techniques are conceptually different from preconditioning. Deflation (in the sense the term is used here) “removes” certain parts from the operator making it singular, while augmentation a...

Gradient iterations for the Rayleigh quotient are elemental methods for computing the smallest eigenvalues of a pair of symmetric and positive definite matrices. A considerable convergence acceleration can be achieved by preconditioning and by computing Rayleigh-Ritz approximations from subspaces of increasing dimensions. An example of the resulting Krylov subspace eigensolvers is the generaliz...