This paper proposes a novel kernel approach to linear dimension reduction for supervised learning. The purpose of the dimension reduction is to find directions in the input space to explain the output as effectively as possible. The proposed method uses an estimator for the gradient of regression function, based on the covariance operators on reproducing kernel Hilbert spaces. In comparison wit...

PAC-Bayes risk bound integrating theories of Bayesian paradigm and structure risk minimization for stochastic classifiers has been considered as a framework for deriving some of the tightest generalization bounds. A major issue in practical use of this bound is estimations of unknown prior and posterior distributions of the concept space. In this paper, by formulating the concept space as Repro...

This paper presents an online support vector machine (SVM) that uses the stochastic meta-descent (SMD) algorithm to adapt its step size automatically. We formulate the online learning problem as a stochastic gradient descent in reproducing kernel Hilbert space (RKHS) and translate SMD to the nonparametric setting, where its gradient trace parameter is no longer a coefficient vector but an eleme...

We propose denotational semantics for a language of probabilistic arithmetic expressions based on reproducing kernel Hilbert spaces (RKHS). The RKHS approach has numerous practical advantages, but from a semantics point of view the most important is ability to provide convergence guarantees on approximate evaluations of expressions. We present preliminary results on convergence bounds, adapting...

We consider reproducing kernels K : ⌦ ⇥ ⌦ ! R in multiscale series expansion form, i.e., kernels of the form K (x, y) = P ` 2N`P j2I`` ,j (x) `,j (y) with weightsànd structurally simple basis functions`,i. Here, we deal with basis functions such as polynomials or frame systems, where, for`2 N, the index set I ` is finite or countable. We derive relations between approximation properties of spac...

We consider modelling policies for MDPs in (vector-valued) reproducing kernel Hilbert function spaces (RKHS). This enables us to work “non-parametrically” in a rich function class, and provides the ability to learn complex policies. We present a framework for performing gradientbased policy optimization in the RKHS, deriving the functional gradient of the return for our policy, which has a simp...

Voice activity detection (VAD) is used to detect whether the acoustic signal belongs to speech or non-speech clusters based on the statistical distribution of the acoustic features. Traditional VAD algorithms are applied in a linear transformed space without any constraint relating to the special characteristics speech or noise. As a result, the VAD algorithms are not robust to noise interferen...