Zipf’s law, which states that the probability of an observation is inversely proportional to its rank, has been observed in many different domains. Although there are models that explain Zipf’s law in each of them, there is not yet a general mechanism that covers all, or even most, domains. Here we propose such a mechanism. It relies on the observation that real world data is often generated fr...

This paper gives an overview of methods for utilizing large process data matrices. These data matrices are almost always of less than full statistical rank, and therefore latent variable methods are shown to be well suited to obtaining useful subspace models from them for treating a variety of important industrial problems. An overview of the important concepts behind latent variable models is ...

To find he precursor of contemporary latent variable modelling one must go back to the beginning of the 20th century and Charles Spearman’s invention of factor analysis. This was followed, half a century later, by latent class and latent trait analysis and, from the 1960’s onwards, by covariance structure analysis. The most recent additions to the family have been in the area of time series. We...

In this contribution, we propose a generic online (also sometimes called adaptive or recursive) version of the Expectation-Maximisation (EM) algorithm applicable to latent variable models of independent observations. Compared to the algorithm of Titterington (1984), this approach is more directly connected to the usual EM algorithm and does not rely on integration with respect to the complete d...

Parameter constraints in generalized linear latent variable models are discussed. Both linear equality and inequality constraints are considered. Maximum likelihood estimators for the parameters of the constrained model and corrected standard errors are derived. A significant reduction in the dimension of the optimization problem is achieved with the proposed methodology for fitting models subj...

This paper takes a discourse-oriented perspective for disambiguating common and proper noun mentions with respect to Wikipedia. Our novel approach models the relationship between disambiguation and aspects of cohesion using Markov Logic Networks with latent variables. Considering cohesive aspects consistently improves the disambiguation results on various commonly used data sets.

We consider a spatial generalized linear latent variable model with and without normality distributional assumption on the latent variables. When the latent variables are assumed to be multivariate normal, we apply a Laplace approximation. To relax the assumption of marginal normality in favor of a mixture of normals, we construct a multivariate density with Gaussian spatial dependence and give...

Latent variable models are used for analyzing multivariate data. Recently, generalized linear latent variable models for categorical, metric, and mixed-type responses estimated via maximum likelihood (ML) have been proposed. Model deviations, such as data contamination, are shown analytically, using the influence function and through a simulation study, to seriously affect ML estimation. This a...

Black box variational inference allows researchers to easily prototype and evaluate an array of models. Recent advances allow such algorithms to scale to high dimensions. However, a central question remains: How to specify an expressive variational distribution that maintains efficient computation? To address this, we develop hierarchical variational models (HVMs). HVMs augment a variational ap...

Unsupervised learning on imbalanced data is challenging because, when given imbalanced data, current model is often dominated by the major category and ignores the categories with small amount of data. We develop a latent variable model that can cope with imbalanced data by dividing the latent space into a shared space and a private space. Based on Gaussian Process Latent Variable Models, we pr...