Competitive neural networks can be used to e ciently quantize image and video data. We discuss a novel class of vector quantizers which perform noise robust data compression. The vector quantizers are trained to simultaneously compensate channel noise and code vector elimination noise. The training algorithm to estimate code vectors is derived by the maximum entropy principle in the spirit of d...

We currently know the following four bounds on rate as a function of relative distance, three of them upper bounds (which tell us what rate-distance combinations are impossible), and one lower bound (which tells us what rate-distance combinations we can achieve). In the following, R is the rate and δ the relative distance of a code. For example, a (n, k, d) code has R = k/n and δ = d/n. Hq(x) i...

A novel source coding scheme based on turbo codes was presented in [1]. Lossless data compression is thereby achieved by puncturing data encoded with a turbo code while checking the integrity of the reconstructed information during compression. In this paper we apply code optimization tools to serially concatenated turbo source codes. The goal of the optimization is to minimize the area of the ...

The idea of this work is to compare a new positive and entropy stable approximate Riemann solver by Francois Bouchut with a state-of the-art algorithm for astrophysical fluid dynamics. We implemented the new Riemann solver into an astrophysical PPM-code, the Prometheus code, and also made a version with a different, more theoretically grounded higher order algorithm than PPM. We present shock t...

In this paper, a novel pre-compression rate-distortion optimization algorithm is proposed, which can reduce computation power and memory requirement of JPEG 2000 encoder. It can reduce the wasted computational power of the entropy coder (EBCOT Tier-1) and unnecessary memory requirement for the code-stream. Distortion and rate of coding passes are calculated and estimated before coding, and ther...

Algorithms for encoding and decoding finite strings over a finite alphabet are described. The coding operations are arithmetic involving rational numbers li as parameters such that Zi2"i 5 2". This coding technique requires no blocking, and the per-symbol length of the encoded string approaches the associated entropy within E . The coding speed is comparable to that of conventional coding metho...

We quantify precisely the distribution of the output of a binary random number generator (RNG) after conditioning with a binary linear code generator matrix by showing the connection between the Walsh spectrum of the resulting random variable and the weight distribution of the code. Previously known bounds on the performance of linear binary codes as entropy extractors can be derived by conside...

In this paper we present online encoding and decoding algorithms for an infinite input alphabet. The well-known encoding algorithm that explicitly constructs the optimal codes is proposed by Huffman. This algorithm is clearly inappropriate for online encoding because the whole input data are not known before starting the process. We propose an algorithm that can be encoded while the input data ...

Complex systems are increasingly being viewed as distributed information processing systems, particularly in the domains of computational neuroscience, bioinformatics and Artificial Life. This trend has resulted in a strong uptake in the use of (Shannon) information-theoretic measures to analyse the dynamics of complex systems in these fields. We introduce the Java Information Dynamics Toolkit ...

In this paper, the two parameter ADK entropy, as a generalized of Re'nyi entropy, is considered and some properties of it, are investigated. We will see that the ADK entropy for continuous random variables is invariant under a location and is not invariant under a scale transformation of the random variable. Furthermore, the joint ADK entropy, conditional ADK entropy, and chain rule of this ent...