We study the natural gradient flow of the expected value Ep [ f ] of an objective function f for p in an exponential family. We parameterize the exponential family with the expectation parameters and we show that the dynamical system associated to the natural gradient flow can be extended outside the marginal polytope.

Three-stage t-tests of separated one-sided hypotheses are derived, extending Lorden’s optimal three-stage tests for the one-dimensional exponential family by using Lai and Zhang’s generalization of Schwarz’s optimal fully-sequential tests to the multiparameter exponential family. The resulting three-stage t-tests are shown to be asymptotically optimal, achieving the same average sample size as ...

 Abstract: A new method to generate various family of distributions is introduced. This method introduces a new two-parameter extension of the exponential distribution to illustrate its application. Some statistical and reliability properties of the new distribution, including explicit expressions for the moments, quantiles, mode, moment generating function, mean residual lifetime, stochas...

We explore the information geometric structure of the statistical manifold generated by the κ-deformed exponential family. The dually-flat manifold is obtained as a dualistic Hessian structure by introducing suitable generalization of the Fisher metric and affine connections. As a byproduct, we obtain the fluctuation-response relations in the κ-formalism based on the κ-generalized exponential f...

We propose a Laplace approximation that creates a stochastic unit from any smooth monotonic activation function, using only Gaussian noise. This paper investigates the application of this stochastic approximation in training a family of Restricted Boltzmann Machines (RBM) that are closely linked to Bregman divergences. This family, that we call exponential family RBM (Exp-RBM), is a subset of t...

We discuss the relationship between the statistical embedding curvature [1, 2] and the logarithmic regret [11] (regret for short) of the Bayesian prediction strategy (or coding strategy) for curved exponential families and Markov models. The regret of a strategy is defined as the difference of the logarithmic loss (code length) incurred by the strategy and that of the best strategy for each dat...

Exponential family extensions of principal component analysis (EPCA) have received a considerable amount of attention in recent years, demonstrating the growing need for basic modeling tools that do not assume the squared loss or Gaussian distribution. We extend the EPCA model toolbox by presenting the first exponential family multi-view learning methods of the partial least squares and canonic...

Exponential family extensions of principal component analysis (EPCA) have received a considerable amount of attention in recent years, demonstrating the growing need for basic modeling tools that do not assume the squared loss or Gaussian distribution. We extend the EPCA model toolbox by presenting the first exponential family multi-view learning methods of the partial least squares and canonic...

Thompson Sampling has been demonstrated in many complex bandit models, however the theoretical guarantees available for the parametric multi-armed bandit are still limited to the Bernoulli case. Here we extend them by proving asymptotic optimality of the algorithm using the Jeffreys prior for 1-dimensional exponential family bandits. Our proof builds on previous work, but also makes extensive u...