We information-theoretically reformulate two measures of capacity from statistical learning theory: empirical VC-entropy and empirical Rademacher complexity. We show these capacity measures count the number of hypotheses about a dataset that a learning algorithm falsifies when it finds the classifier in its repertoire minimizing empirical risk. It then follows from that the future performance o...

In this paper we discuss the entropy and information aspects of a living cell. Particular attention is paid to the information gain on assembling and maintaining a living state. Numerical estimates of the information and entropy reduction are given and discussed in the context of the cell's metabolic activity. We discuss a solution to an apparent paradox that there is less information content i...

This paper describes a fast, low-complexity, entropy efficient video coder for wavelet pyramids. This coder approaches the entropy-limited coding rate of video wavelet pyramids, is fast in both hardware and software implementations, and has low complexity (no multiplies) for use in ASICs. It consists of a modified Z-coder used to code the zero/non-zero significance function and Huffman coding f...

In this correspondence, we give a new definition of generalized Hamming weights of nonlinear codes and a new interpretation connected with it. These generalized weights are determined by the entropy/length profile of the code. We show that this definition characterizes the performance of nonlinear codes on the wire-tap channel of type II. The new definition is invariant under translates of the ...

A near-lossless image compression scheme is presented. It is essentially a differential pulse code modulation (DPCM) system with a mechanism incorporated to minimize the entropy of the quantized prediction error sequence. With a "near-lossless" criterion of no more than a d gray-level error for each pixel, where d is a small nonnegative integer, trellises describing all allowable quantized pred...

the mechanical aspect of entropy-exergy relationship, together with the thermal aspect usually consid-ered, leads to an extended formulation of physical exergy based on both maximum useful work and maximum useful heat that are the outcome of available energy of a thermodynamic system. this approach suggests that a mechanical entropy can be studied, in addition to the already used thermal entrop...

Source coding is accomplished via the mapping of consecutive source symbols (blocks) into code blocks of fixed or variable length. The fundamental limits in source coding introduces a tradeoff between the rate of compression and the fidelity of the recovery. However, in practical communication systems many issues such as computational complexity, memory capacity, and memory access requirements ...

New lower and upper bounds are obtained for the compression of optimal binary prefix codes according to various nonlinear codeword length objectives. Like the coding bounds for Huffman coding — which concern the traditional linear code objective of minimizing average codeword length — these are in terms of a form of entropy and the probability of the most probable input symbol. As in Huffman co...