Machine Learning can be divided into two schools of thought: generative model learning and discriminative model learning. While the MCS community has been focused mainly on the latter, our paper is concerned with questions that arise from ensembles of generative models. Generative models provide us with neat ways of thinking about two interesting learning issues: model selection and semi-superv...

An interesting question surrounding semisupervised learning for NLP is: should we use discriminative models or generative models? Despite the fact that generative models have been frequently employed in a semi-supervised setting since the early days of the statistical revolution in NLP, we advocate the use of discriminative models. The ability of discriminative models to handle complex, high-di...

We describe a method for incrementally constructing a hierarchical generative model of an ensemble of binary data vectors. The model is composed of stochastic, binary, logistic units. Hidden units are added to the model one at a time with the goal of minimizing the information required to describe the data vectors using the model. In addition to the top-down generative weights that define the m...

The past several years have seen remarkable progress in generative models which produce convincing samples of images and other modalities. A shared component of many powerful generative models is a decoder network, a parametric deep neural net that defines a generative distribution. Examples include variational autoencoders, generative adversarial networks, and generative moment matching networ...

For learning meaningful representations of data, a rich source of prior knowledge may come in the form of a generative black box, e.g. a graphics program that generates realistic facial images. We consider the problem of learning the inverse of a given generative model from data. The problem is non-trivial because it is difficult to create labelled training cases by hand, and the generative map...

The established definition of replication in terms of the conditions of causality, similarity and information transfer is very broad. We draw inspiration from the literature on self-reproducing automata to strengthen the notion of information transfer in replication processes. To the triple conditions of causality, similarity and information transfer, we add a fourth condition that defines a “g...

It has been shown that generative representations, which allow the reuse of code, perform well on problems with high regularity (i.e. where a phenotypic motif must be repeated many times). To date, however, generative representations have not been tested on irregular problems. It is unknown how they will fare on problems with intermediate and low amounts of regularity. This paper compares a gen...

We consider the problem of learning deep generative models from data. We formulate a method that generates an independent sample via a single feedforward pass through a multilayer preceptron, as in the recently proposed generative adversarial networks (Goodfellow et al., 2014). Training a generative adversarial network, however, requires careful optimization of a difficult minimax program. Inst...

The convolutional neural networks (CNNs) have proven to be a powerful tool for discriminative learning. Recently researchers have also started to show interest in the generative aspects of CNNs in order to gain a deeper understanding of what they have learned and how to further improve them. This paper investigates generative modeling of CNNs. The main contributions include: (1) We construct a ...

This paper investigates the role of architectures in information infrastructures. We find that a key dimension is lacking in the extant architectural theorizing: the relations between an IIs’ architecture and structures in the user and development organizations/communities. Further, we find that an II’s architecture needs to change over time. We suggest the concept of generative architecture to...