This paper proposes a distributionally robust approach to logistic regression. We use the Wasserstein distance to construct a ball in the space of probability distributions centered at the uniform distribution on the training samples. If the radius of this ball is chosen judiciously, we can guarantee that it contains the unknown datagenerating distribution with high confidence. We then formulat...

This chapter describes a tree-structured extension and generalization of the logistic regression method for fitting models to a binary-valued response variable. The technique overcomes a significant disadvantage of logistic regression, which is interpretability of the model in the face of multicollinearity and Simpson’s paradox. Section 1 summarizes the statistical theory underlying the logisti...

This paper addresses the important tradeoff between privacy and learnability, when designing algorithms for learning from private databases. We focus on privacy-preserving logistic regression. First we apply an idea of Dwork et al. [6] to design a privacy-preserving logistic regression algorithm. This involves bounding the sensitivity of regularized logistic regression, and perturbing the learn...

Semi-supervised learning has recently emerged as a new paradigm in the machine learning community. It aims at exploiting simultaneously labeled and unlabeled data for classification. We introduce here a new semi-supervised algorithm. Its originality is that it relies on a discriminative approach to semisupervised learning rather than a generative approach, as it is usually the case. We present ...

In this paper, we consider the logistic regression model with an integrated regressor driven by a general linear process. In particular, we derive the limit distributions of the nonlinear least squares (NLS) estimators and their t-ratios of the parameters in the model. It is shown that the NLS estimators are generally not efficient. Moreover, the t-ratios for the level parameters have limit dis...

Based on the experience of teaching logistic regression to non-mathematicians, a number of areas of possible confusion are identified that may arise particularly when the method is contrasted with multiple linear regression. The fact that the model is multiplicative in odds ratios means that the concept of interaction needs to be clearly defined. Confidence intervals for the estimates of the od...

We extend logistic regression by using t-exponential families which were introduced recently in statistical physics. We examine our algorithm for both binary classfication and multiclass classfication with both L1 and L2 regularizer. The objective function of our algorithm is non-convex, an efficient block coordinate descent optimization scheme is derived for estimating the parameters. Because ...

Bayesian analyses of multivariate binary or categorical outcomes typically rely on probit or mixed effects logistic regression models that do not have a marginal logistic structure for the individual outcomes. In addition, difficulties arise when simple noninformative priors are chosen for the covariance parameters. Motivated by these problems, we propose a new type of multivariate logistic dis...

Learning from data streams is a well researched task both in theory and practice. As remarked by Clarkson, Hazan and Woodru [12], many classi cation problems cannot be very well solved in a streaming setting. For previous model assumptions, there exist simple, yet highly arti cial lower bounds prohibiting space e cient onepass algorithms. At the same time, several classi cation algorithms are o...