Bayesian Analysis of Between-Group Differences in Variance Components in Hierarchical Generalized Linear Models

Frequentist approaches to making inferences about the variances of random cluster effects in hierarchical generalized linear models (HGLMs) for non-normal variables have several limitations. These include reliance on asymptotic theory, questionable properties of classical likelihood ratio tests when pseudo-likelihood methods are used for estimation, and a failure to account for uncertainty in the estimation of features of prior distributions for model parameters. This paper compares and contrasts alternative approaches to making a specific type of inference about the variance components in an HGLM, focusing on the difference in variance components between two independent groups of clusters. A Bayesian approach to making inferences about these types of differences is proposed that circumvents many of the problems associated with alternative frequentist approaches. The Bayesian approach and alternative frequentist approaches are applied to an analysis of real survey data collected in the Continuous National Survey of Family Growth (NSFG). The primary analytic question of interest concerns differences in the variances of random interviewer effects between two independent groups of interviewers, which may indicate that particular subsets of interviewers are having adverse effects on the quality of the survey data. Inferences regarding differences in interviewer variance components are shown to vary depending on the approach taken, with significant differences suggested by problematic frequentist analyses no longer evident when applying more appropriate Bayesian analysis methods.