Let AN N be the set of the Gram matrices of systems fejg N j formed by vectors ej of a Hilbert spaceH with norms kejkH j N Let BN K be the set of the Gram matrices of systems ffjg N j formed by functions fj L with kfjkL K j N It is shown that for any K the set BN K is narrower than AN as N More precisely it is proved that not every matrix A in AN can be represented as A B where B BN K and is a ...

We define the discrete norm of a complex m× n matrix A by ‖A‖∆ := max 06=ξ∈{0,1}n ‖Aξ‖ ‖ξ‖ , and show that c √ log h(A) + 1 ‖A‖ ≤ ‖A‖∆ ≤ ‖A‖, where c > 0 is an explicitly indicated absolute constant, h(A) = √ ‖A‖1‖A‖∞/‖A‖, and ‖A‖1, ‖A‖∞, and ‖A‖ = ‖A‖2 are the induced operator norms of A. Similarly, for the discrete Rayleigh norm ‖A‖P := max 06=ξ∈{0,1} 06=η∈{0,1} |ξAη| ‖ξ‖‖η‖ we prove the esti...

The concepts of paracontracting, pseudocontracting and nonexpanding operators have been shown to be useful in proving convergence of asynchronous or parallel iteration algorithms. The purpose of this paper is to give characterizations of these operators when they are linear and finite-dimensional. First we show that pseudocontractivity of stochastic matrices with respect to ‖ · ‖∞ is equivalent...

Limited memory quasi-Newton methods and trust-region methods represent two efficient approaches used for solving unconstrained optimization problems. A straightforward combination of them deteriorates the efficiency of the former approach, especially in the case of large-scale problems. For this reason, the limited memory methods are usually combined with a line search. We show how to efficient...

Volterra difference equations arise in the mathematical modeling of some real phenomena, and also in numerical schemes for solving differential and integral equations (cf. [7, 8] and the references therein). One of the basic methods in the theory of stability and boundedness of Volterra difference equations is the direct Lyapunov method (see [1, 3, 4] and the references therein). But finding th...

Many of the most common and useful properties of Sobolev spaces defined over a domain (open set) in Euclidean space require that the domain has a minimal degree of regularity. To this end, the domain is often assumed to satisfy a “cone condition.” For example, various imbeddings of Sobolev spaces into Lebesgue spaces or spaces of bounded continuous functions (the Sobolev imbedding theorem), and...

Using sparse-inducing norms to learn robust models has received increasing attention from many fields for its attractive properties. Projection-based methods have been widely applied to learning tasks constrained by such norms. As a key building block of these methods, an efficient operator for Euclidean projection onto the intersection of l1 and l1,q norm balls (q = 2 or ∞) is proposed in this...

Regularization for matrix factorization (MF) and approximation problems has been carried out in many different ways. Due to its popularity in deep learning, dropout has been applied also for this class of problems. Despite its solid empirical performance, the theoretical properties of dropout as a regularizer remain quite elusive for this class of problems. In this paper, we present a theoretic...

Let A be a matrix whose columns X1, . . . ,XN are independent random vectors in Rn. Assume that the tails of the 1-dimensional marginals decay as P(| 〈Xi, a〉 | ≥ t) ≤ t−p uniformly in a ∈ Sn−1 and i ≤ N . Then for p > 4 we prove that with high probability A/ √ n has the Restricted Isometry Property (RIP) provided that Euclidean norms |Xi| are concentrated around √ n. We also show that the covar...

We discuss generalized proximity operators (GPO) and their associated generalized projection problems. On inputs of size n, we show how to efficiently apply GPOs and generalized projections for separable norms and distance-like functions to accuracy in O(n log(1/ )) time. We also derive projection algorithms that run theoretically in O(n log n log(1/ )) time but can for suitable parameter range...