It has been recently shown that, under the margin (or low noise) assumption, there exist classifiers attaining fast rates of convergence of the excess Bayes risk, i.e., the rates faster than n−1/2. The works on this subject suggested the following two conjectures: (i) the best achievable fast rate is of the order n−1, and (ii) the plug-in classifiers generally converge slower than the classifie...

We propose a novel framework for the differentially private ERM, input perturbation. Existing differentially private ERM implicitly assumed that the data contributors submit their private data to a database expecting that the database invokes a differentially private mechanism for publication of the learned model. In input perturbation, each data contributor independently randomizes her/his dat...

Learnability has always been one of the most central problems in learning theory. Most previous studies on this issue were based on the assumption that the samples are drawn independently and identically according to an underlying (unknown) distribution. The i.i.d. assumption, however, does not hold in many real applications. In this paper, we study the learnability of problems where the sample...

We introduce and analyze Discriminative State-Space Models for forecasting nonstationary time series. We provide data-dependent generalization guarantees for learning these models based on the recently introduced notion of discrepancy. We provide an in-depth analysis of the complexity of such models. We also study the generalization guarantees for several structural risk minimization approaches...

Experimental determination of magnetization characteristics of switched reluctance motors (SRM's) is quite important in their accurate performance prediction. Over the last decade, various experimental procedures have been used to obtain these characteristics. Every evolved new method has its own limitations and constraints. This paper describes an improved, simple and cost effective experiment...

Recall that: L(w) = 1 n E‖Xw − Y ‖ = 1 n E‖Xw − E[Y ]‖ + σ Define our “empirical loss” as: L̂(w) = 1 n ‖Xw − Y ‖ which has no expectation over Y . Note that for a fixed w E[L̂(w)] = L(w) e.g. the empirical loss is an unbiased estimate of the true loss. Suppose we knew the support size q. One algorithm is to simply find the estimator which minimizes the empirical loss and has support only on q coo...

Following the results in Rüschendorf and Uckelmann (2002), we introduce the completely mixable distributions on R and prove that distributions with monotone density and moderate mean are completely mixable. Using this method we solve the minimization problem minXi∼P Ef(X1 + · · ·+ Xn) for convex functions f and marginal distributions P with monotone density. Our results also provide valuable im...

We present novel minibatch stochastic optimization methods for empirical risk minimization problems, the methods efficiently leverage variance reduced first-order and sub-sampled higherorder information to accelerate the convergence speed. For quadratic objectives, we prove improved iteration complexity over state-of-the-art under reasonable assumptions. We also provide empirical evidence of th...

The representer theorem assures that kernel methods retain optimality under penalized empirical risk minimization. While a sufficient condition on the form of the regularizer guaranteeing the representer theorem has been known since the initial development of kernel methods, necessary conditions have only been investigated recently. In this paper we completely characterize the necessary and suf...