In this paper we study the problem of minimizing the average of a large number (n) of smooth convex loss functions. We propose a new method, S2GD (Semi-Stochastic Gradient Descent), which runs for one or several epochs in each of which a single full gradient and a random number of stochastic gradients is computed, following a geometric law. The total work needed for the method to output an ε-ac...

We design a stochastic algorithm to train any smooth neural network to ε-approximate local minima, using O(ε−3.25) backpropagations. The best result was essentially O(ε−4) by SGD. More broadly, it finds ε-approximate local minima of any smooth nonconvex function in rate O(ε−3.25), with only oracle access to stochastic gradients. ∗V1 appeared on arXiv on this date. V2 and V3 polished writing. Th...

In a recent Letter [1] Öğüt, Chelikowsky, and Louie (OCL) calculated the optical gap of Si nanocrystals as ε g,OCL = ε qp g,LDA − E e−h Coul, where ε qp g,LDA is the quasi-particle gap in the local-density approximation (LDA) and E Coul is the electron-hole Coulomb energy. They argued that their method produces different results from conventional approaches (e.g. pseudopotential [2]). We show i...

We present a novel feature selection algorithm for the k-means clustering problem. Our algorithm is randomized and, assuming an accuracy parameter ε ∈ (0, 1), selects and appropriately rescales in an unsupervised manner Θ(k log(k/ε)/ε) features from a dataset of arbitrary dimensions. We prove that, if we run any γ-approximate k-means algorithm (γ ≥ 1) on the features selected using our method, ...

Decomposition methods are used to convert general constraint satisfaction problems into an equivalent tree-structured problem that can be solved more effectively. Recently, diagnosis algorithms for treestructured systems have been introduced, but the prerequisites of coupling these algorithms to the outcome of decomposition methods have not been analyzed in detail, thus limiting their diagnosti...

I shortly review the present status of the theoretical estimates of ε/ε . I consider a few aspects of the theoretical calculations which may be relevant in understanding the present experimental results. I discuss the role of higher order chiral corrections and in general of non-factorizable contributions for the explanation of the ∆I = 1/2 selection rule in kaon decays and ε/ε . Lacking reliab...

A direct method called fixation is introduced for solving the general motion vision problem, arbitrary motion relative to an arbitrary environment. This method results in a linear constraint equation which explicitly expresses the rotational velocity in terms of the translational velocity. The combination of this constraint equation with the Brightness-Change Constraint Equation solves the gene...

Abstract: An interior point method for solving nonlinear multiobjective programming problems, over a convex set contained in the real space R, has been developed in this paper. In this method a new strictly concave logarithmic barrier function has been suggested in order to transform the orginal problem into a sequence of unconstrained subproblems. These subproblems can be solved using Newton m...