In this paper, we consider the logistic regression model with an integrated regressor driven by a general linear process. In particular, we derive the limit distributions of the nonlinear least squares (NLS) estimators and their t-ratios of the parameters in the model. It is shown that the NLS estimators are generally not efficient. Moreover, the t-ratios for the level parameters have limit dis...

Based on the experience of teaching logistic regression to non-mathematicians, a number of areas of possible confusion are identified that may arise particularly when the method is contrasted with multiple linear regression. The fact that the model is multiplicative in odds ratios means that the concept of interaction needs to be clearly defined. Confidence intervals for the estimates of the od...

We extend logistic regression by using t-exponential families which were introduced recently in statistical physics. We examine our algorithm for both binary classfication and multiclass classfication with both L1 and L2 regularizer. The objective function of our algorithm is non-convex, an efficient block coordinate descent optimization scheme is derived for estimating the parameters. Because ...

Bayesian analyses of multivariate binary or categorical outcomes typically rely on probit or mixed effects logistic regression models that do not have a marginal logistic structure for the individual outcomes. In addition, difficulties arise when simple noninformative priors are chosen for the covariance parameters. Motivated by these problems, we propose a new type of multivariate logistic dis...

Learning from data streams is a well researched task both in theory and practice. As remarked by Clarkson, Hazan and Woodru [12], many classi cation problems cannot be very well solved in a streaming setting. For previous model assumptions, there exist simple, yet highly arti cial lower bounds prohibiting space e cient onepass algorithms. At the same time, several classi cation algorithms are o...

This article proposes a method for multiclass classification problems using ensembles of multinomial logistic regression models. A multinomial logit model is used as a base classifier in ensembles from random partitions of predictors. The multinomial logit model can be applied to each mutually exclusive subset of the feature space without variable selection. By combining multiple models the pro...

In binary classification problems it is common for the two classes to be imbalanced: one case is very rare compared to the other. In this paper we consider the infinitely imbalanced case where one class has a finite sample size and the other class’s sample size grows without bound. For logistic regression, the infinitely imbalanced case often has a useful solution. Under mild conditions, the in...

In this report, several experiments have been conducted on a spam data set with Logistic Regression based on Gradient Descent approach. First, the overfitting effect is shown with basic settings (vanilla version). Then Stochastic Gradient Descent and 2-Norm Regularization techniques are both implemented with demonstration of the benefits of these two methods in preventing overfitting. Besides, ...

We introduce Graph-Sparse Logistic Regression, a new algorithm for classification for the case in which the support should be sparse but connected on a graph. We validate this algorithm against synthetic data and benchmark it against L1-regularized Logistic Regression. We then explore our technique in the bioinformatics context of proteomics data on the interactome graph. We make all our experi...