This article presents our recent work for participation in the Second International Chinese Word Segmentation Bakeoff. Our system performs two procedures: Out-ofvocabulary extraction and word segmentation. We compose three out-of-vocabulary extraction modules: Character-based tagging with different classifiers – maximum entropy, support vector machines, and conditional random fields. We also co...

Based on the concept of entropy, a new approach to analyse the quality of features as used in speech recognition is proposed. We regard the relation between the hidden Markov model (HMM) states and the corresponding frame based feature vectors as a coding problem, where the states are sent through a noisy recognition channel and received as feature vectors. Using the relation between Shannon’s ...

Anomaly detection is an essential component of the protection mechanisms against novel attacks. In this paper, we propose to use several information-theoretic measures, namely, entropy, conditional entropy, relative conditional entropy, information gain, and information cost for anomaly detection. These measures can be used to describe the characteristics of an audit data set, suggest the appro...

This papers proposes a fast algorithm for estimating the mutual information, difference score function, conditional score and conditional entropy, in possibly high dimensional space. The idea is to discretise the integral so that the density needs only be estimated over a regular grid, which can be done with little cost through the use of a cardinal spline kernel estimator. Score functions are ...

This paper introduces an extension of entropy constrained residual vector quantization (VQ) where intervector dependencies are exploited. The method, which we call conditional entropy-constrained residual VQ, employs a high-order entropy conditioning strategy that captures local information in the neighboring vectors. When applied to coding images, the proposed method is shown to achieve better...

The Rate Constrained Block Matching Algorithm, (RC-BMA), introduced in this paper, jointly minimizes Displaced Frame Di erence (DFD) variance and entropy or conditional entropy of motion vectors for determining the motion vectors. It is intended for use in low rate video coding applications where the contribution of the motion vector rate to the overall coding rate might be signi cant. The DFD ...

This paper studies bivariate distributions with fixed marginals from an information-theoretic perspective. In particular, continuity and related properties of various information measures (Shannon entropy, conditional entropy, mutual information, Rényi entropy) on the set of all such distributions are investigated. The notion of minimum entropy coupling is introduced, and it is shown that it de...