Minimum Description Length Model Selection Criteria for Generalized Linear Models

This paper derives several model selection criteria for generalized linear models (GLMs) following the principle of Minimum Description Length (MDL). We focus our attention on the mixture form of MDL. Normal or normal-inverse gamma distributions are used to construct the mixtures, depending on whether or not we choose to account for possible over-dispersion in the data. For the latter, we use Efron's (1986) double exponential family characterization of GLMs as the likelihood. Standard Laplace approximations are employed to derive computationally tractable selection rules. Each of the criteria we construct have adaptive penalties on model complexity, either explicitly or implicitly. Theoretical results for the normal linear model, and a set of simulations for logistic regression illustrate that mixture MDL can \bridge" AIC and BIC in the sense that it can mimic the performance of either criterion, depending on which is best for the situation at hand.