Minimum Message Length (MML) is a popular method for statistical inference, belonging to the Minimum Description Length (MDL) family. It is a general name for any of several computationally-feasible approximations to the generally NP-Hard Strict Minimum Message Length (SMML) estimator. One often-cited showcase for the power of MML is the Neyman-Scott estimation problem, where most popular estim...

The relationship between the Bayesian approach and the minimum description length approach is established. We sharpen and clarify the general modeling principles minimum description length (MDL) and minimum message length (MML), abstracted as the ideal MDL principle and defined from Bayes’s rule by means of Kolmogorov complexity. The basic condition under which the ideal principle should be app...

Nordhausen and Langley’s IDS is a system for automated integrated scientific discovery. IDS’ is-a hierarchy both organizes knowledge and constrains search. This search bias, however, limits what may be discovered to knowledge learnable by operators that execute local tree manipulations on the is-a hierarchy. I present an alternative approach which uses representation reduction as the driving bi...

I consider the problem of learning an optimal path graphical model from data and show the problem to be NP-hard for the maximum likelihood and minimum description length approaches and a Bayesian approach. This hardness result holds despite the fact that the problem is a restriction of the polynomially solvable problem of nding the optimal tree graphical model.

Inspired by the information theoretic idea of minimum description length, we add a term to the back propagation cost function that penalizes network complexity. We give the details of the procedure, called weight-elimination, describe its dynamics, and clarify the meaning of the parameters involved. From a Bayesian perspective, the complexity term can be usefully interpreted as an assumption ab...

This paper describes a new approach to the analysis of Poisson point processes, in time (1D) or space (2D), which is based on the minimum description length (MDL) framework. Specifically, we describe a fully unsupervised recursive segmentation algorithm for 1D and 2D observations. Experiments illustrate the good performance of the proposed methods.

A hidden Markov model is introduced for descriptive modelling the mosaic–like structures of haplotypes, due to iterated recombinations within a population. Methods using the minimum description length principle are given for fitting such models to training data. Possible applications of the models are delineated, and some preliminary analysis results on real sets of haplotypes are reported, dem...

How can we succinctly describe a million-node graph with a few simple sentences? How can we measure the ‘importance’ of a set of discovered subgraphs in a large graph? These are exactly the problems we focus on. Our main ideas are to construct a ‘vocabulary’ of subgraph-types that often occur in real graphs (e.g., stars, cliques, chains), and from a set of subgraphs, find the most succinct desc...

In this paper, we discuss the difference in code lengths between the code based on the minimum description length (MDL) principle (the MDL code) and the Bayes code under the condition that the same prior distribution is assumed for both codes. It is proved that the code length of the Bayes code is smaller than that of the MDL code by (1) or (1) for the discrete model class and by (1) for the pa...

An improved method for deformable shape-based image segmentation is described. Image regions are merged together and/or split apart, based on their agreement with an a priori distribution on the global deformation parameters for a shape template. Perceptually-motivated criteria are used to determine where/how to split regions, based on the local shape properties of the region group’s bounding c...