The aim of this paper is to show that spatial coupling can be viewed not only as a means to build better graphical models, but also as a tool to better understand uncoupled models. The starting point is the observation that some asymptotic properties of graphical models are easier to prove in the case of spatial coupling. In such cases, one can then use the so-called interpolation method to tra...

Statistical language models are powerful tools which have been used for many tasks within natural language processing. Recently, they have been used for other sequential data such as source code. (Ray et al., 2015) showed that it is possible train an n-gram source code language mode, and use it to predict buggy lines in code by determining “unnatural” lines via entropy with respect to the langu...

We consider the use of second order statistics in two-stage universal source coding. (Examples of two-stage universal codes include the weighted universal vector quantization (WWQ) 12, 3, 41, weighted universal bit allocation (WUBA) [5, 41, and weighted universal transform coding (WUTC) [6, 41 algorithms.) The second order statistics are incorporated in two-stage universal source codes in a man...

where l(x) is the length of codeword c(x) for a symbol x ∈ X , and p(x) is the probability of the symbol. Intuitively, a good code should preserve the information content of an outcome. Since information content depends on the probability of the outcome (it is higher if probability is lower, or equivalently if the outcome is very uncertain), a good codeword will use fewer bits to encode a certa...

We present a simple computational approach to assigning a measure of complexity and information/entropy to families of natural languages, based on syntactic parameters and the theory of error correcting codes. We associate to each language a binary string of syntactic parameters and to a language family a binary code, with code words the binary string associated to each language. We then evalua...

The authentication code (A-code) is the one of the most fundamental cryptographic protocols in information-theoretic cryptography, and it provides information-theoretic integrity or authenticity, i.e., preventing information from being altered or substituted by the adversary having unbounded computational powers. In addition, it has a wide range of applications such as multiparty computations a...

Exercise 1. Design a Huffman code for the outcome of a fair die. Evaluate the average length, and compare it against the source entropy. For which source distribution this would be an optimal code? Exercise 2. Let p be the probability of head in a sequence of tosses of a biased coin. A sequence of messages is parsed using the alphabet X = {h, th, tt}. Evaluate H(X) in bits/message and also in b...

This paper applies an exergy-based method on the two-dimensional airfoil and various threedimensional wing planforms to investigate the impact of exergy on aerodynamic designs. Our intent here is to report our efforts to couple a numerical code with an exergy analysis, a shape shifting algorithm and an optimization code to achieve desired airfoil and wing shapes. Exergy-based optimization is pe...

The objective of the present communication is to develop new genuine exponentiated mean codeword lengths and to study deeply the problem of correspondence between well known measures of entropy and mean codeword lengths. With the help of some standard measures of entropy, we have illustrated such a correspondence. In literature, we usually come across many inequalities which are frequently used...