Recently, we introduced a simple variational bound on mutual information, that resolves some of the difficulties in the application of information theory to machine learning. Here we study a specific application to Gaussian channels. It is well known that PCA may be viewed as the solution to maximizing information transmission between a high dimensional vector x and its low dimensional represen...

In this paper, we revisit the problem of finding the average capacity of the Gaussian feedback channel. First, we consider the problem of finding the average capacity of the analog colored Gaussian noise channel where the noise has an arbitrary spectral density. We introduce a new approach to the problem where we solve the problem over a finite number of transmissions and then consider the limi...

We consider a linear Gaussian noise channel used with delayed feedback. The channel noise is assumed to be an ARMA (autoregressive and/or moving average) process. We reformulate the Gaussian noise channel into an intersymbol interference channel with white noise, and show that the delayed-feedback of the original channel is equivalent to the instantaneous-feedback of the derived channel. By gen...

Major principles of the method of the linear codes multithreshhold decoding as search for the global functional extremum for a great number of variables are considered. It was demonstrated that the multithreshold decoding efficiency is close to the results ensured by the optimum exhaustive search methods. Decoding complexity under software and hardware realizations is evaluated.

We investigate the use of multiple transmitting and or receiving antennas for single user communications over the additive Gaussian channel with and without fading We derive formulas for the capacities and error exponents of such channels and describe computational procedures to evaluate such for mulas We show that the potential gains of such multi antenna systems over single antenna systems is...

We examine codes, over the additive Gaussian noise channel, designed for for reliable communication at some specific signal-to-noise ratio (snr) and constrained by the permitted MMSE at some lower snr. We show that the maximum possible rate is the one attained by superposition codebooks. Moreover, the MMSE function of codes attaining this maximum rate under the MMSE constraint is completely def...

This paper considers the problem of channel coding over Gaussian intersymbol interference (ISI) channels with a given metric decoding rule. Specifically, it is assumed that the mismatched decoder has an incorrect assumption on the impulse response function. The mismatch capacity is the highest achievable rate for a given decoding rule. Existing lower bounds to the mismatch capacity for channels...

In this work, we introduce the notion of the error exponent region for a multi-user channel. This region specifies the set of error-exponent vectors that are simultaneously achievable by all users in the multi-user channel. This is done by associating different probabilities of error for different users, contrary to the traditional approach where a single probability of system error is consider...

In this overview paper, we introduce an application of stochastic orders in wireless communications. In particular, we show how to use stochastic orders to investigate the ergodic capacity results for fast fading Gaussian memoryless multiuser channels when only the statistics of the channel state information are known at the transmitters (CSIT). In general, the characterization of the capacity ...

In this thesis we study the design of universal codes for parallel Gaussian channels with 2 sub-channels present. We study the universality both in terms of the uncertainty in the relative quality of the two sub-channels for a fixed maximum rate, C*, and in terms of the uncertainty of the achievable maximum rate. In our architecture, we will convert the parallel Gaussian channel into a set of s...