We emphasize the importance of variance reduction in stochastic methods and propose a probabilistic interpretation as a way to store information about past gradients. The resulting algorithm is very similar to the momentum method, with the difference that the weight over past gradients depends on the distance moved in parameter space rather than the number of steps.

We introduce covariance reducing models for studying the sample covariance matrices of a random vector observed in different populations. The models are based on reducing the sample covariance matrices to an informational core that is sufficient to characterize the variance heterogeneity among the populations. They possess useful equivariance properties and provide a clear alternative to spectr...

Decision tree grafting adds nodes to an existing decision tree with the objective of reducing prediction error. A new grafting algorithm is presented that considers one set of training data only for each leaf of the initial decision tree, the set of cases that fail at most one test on the path to the leaf. This new technique is demonstrated to retain the error reduction power of the original gr...

We develop a variance reduction method for smoothing splines. We do this by showing that the quadratic interpolation method introduced in Cheng et al. (2006), for local linear estimators, also works for smoothing splines. For a given point of estimation, Cheng et al. (2006) define a variance-reduced local linear estimate as a linear combination of classical estimates at three nearby points. We ...

Multi-fold cross-validation is an established practice to estimate the error rate of a learning algorithm. Quantifying the variance reduction gains due to cross-validation has been challenging due to the inherent correlations introduced by the folds. In this work we introduce a new and weak measure called loss stability and relate the cross-validation performance to this measure; we also establ...

In this paper, we propose some novel normalization and fusion techniques for biometric matching score level fusion in person verification. While conventional matching score level fusion methods use global score statistics, we consider in this work both genuine and impostor statistics separately. Performing a joint mean normalization of the separate monomodal scores, multimodal scores with less ...

Variance within the manufacturing system leads to uneven shop loads, long manufacturing lead times, and unreliable customer service. This study compares techniques that reduce system variance to techniques that manage system variance. The study is placed in a dual resource constrained job shop. Results indicate that reducing system variance improves flow time and customer service performance me...

With the objective of supporting a more precise formulation of this question, and the discussion of the relevant issues, we begin with a formal definition of “language model”. We then propose a description of what “the Language Modeling approach” can be thought to consist of, in the context of IR research; first with a strict interpretation in mind, and then with a more informal view. We conclu...