Context Free Grammars (CFGs) are widely used in programming language descriptions, natural language processing, compilers, and other areas of software engineering where there is a need for describing the syntactic structures of programs. Grammar inference (GI) is the induction of CFGs from sample programs and is a challenging problem. We describe an unsupervised GI approach which uses simplicit...

This paper reviews the principle of Minimum Description Length (MDL) for problems of model selection. By viewing statistical modeling as a means of generating descriptions of observed data, the MDL framework discriminates between competing models based on the complexity of each description. This approach began with Kolmogorov’s theory of algorithmic complexity, matured in the literature on info...

One of the central problems in statistics and machine learning is regression: Given values of input variables, called features, develop a model for an output variable, called a response or task. In many settings, there are potentially thousands of possible features, so that feature selection is required to reduce the number of predictors used in the model. Feature selection can be interpreted i...

ong Chul Ye orea Advanced Institute of Science and Technology KAIST epartment of Bio and Brain Engineering io Imaging Signal Processing Laboratory 73-1 Guseong-dong useong-gu, Daejeon 305-701 orea Abstract. Near-infrared spectroscopy NIRS can be employed to investigate brain activities associated with regional changes of the oxyand deoxyhemoglobin concentration by measuring the absorption of ne...

Nonparametric statistical methods developed for analyzing data for high numbers of genes, SNPs, or other biological features tend to overfit data with smaller numbers of features such as proteins, metabolites, or, when expression is measured with conventional instruments, genes. For this medium-scale inference problem, the minimum description length (MDL) framework quantifies the amount of info...

The minimum description length approach can automatic solve the point correspondence problem and give the better statistical shape models than those built by hand or equally spaced way. The current mdl approaches build the models only based on the segmented shapes without considering the local image structure and may place the markers at wrong places. This paper adds Cartesian differential inva...

Automatic word segmentation is a basic requirement for unsupervised learning in morphological analysis. In this paper, we formulate a novel recursive method for minimum description length (MDL) word segmentation, whose basic operation is resegmenting the corpus on a prefix (equivalently, a suffix). We derive a local expression for the change in description length under resegmentation, i.e., one...

In this paper, we present an approach for 2{D shape description based on the Minimum-Description-Length (MDL) criterion. Using this criterion, we can derive, for a given data set and a class of models, a description which best explains the data. The problems of contour partitioning, colinear and cocurvilinear segment grouping, and classiication of 2{D contours in terms of straight and curved, a...

We give a development of the theory of lossy data compression from the point of view of statistics. This is partly motivated by the enormous success of the statistical approach in lossless compression, in particular Rissanen’s celebrated Minimum Description Length (MDL) principle. A precise characterization of the fundamental limits of compression performance is given, for arbitrary data source...

We present a minimalist, unsupervised learning model that induces relatively clean phrasal inversion transduction grammars by employing the minimum description length principle to drive search over a space defined by two opposing extreme types of ITGs. In comparison to most current SMT approaches, the model learns a very parsimonious phrase translation lexicons that provide an obvious basis for...