We consider the analysis of probability distributions through their associated covariance operators from reproducing kernel Hilbert spaces. show that von Neumann entropy and relative these are intimately related to usual notions Shannon entropy, share many properties. They come together with efficient estimation algorithms various oracles on distributions. also product spaces for tensor kernels...

Information theory is concerned with two main tasks. The first task is called data compression (source coding). This is concerned with removing redundancy from data so it can be represented more compactly (either exactly, in a lossless way, or approximately, in a lossy way). The second task is error correction (channel coding), which means encoding data in such a way that it is robust to errors...

when in doubt, ask information theory when you feel confident, ask anyway dedicated to my wife, Ilia

Information retrieval is concerned with how to classify information and how to judge the similarity between two objects, such aswritten documents. As the amount of information available in digital formhas grown, so too has theneed for accurate and scalable algorithms for handling this information. Information theory is concerned with the production and transmission of information. Using a frame...