In the last lecture, we proved the positive part of Shannon's capacity theorem for the BSC. We showed that by the probabilistic method, there exists an encoding function E and a decoding function D such that E m Pr noise e of BSCp [D(E(m) + e) = m] ≤ 2 −δ ′ n. (1) In other words, the average decoding error probability is small. However, we need to show that the maximum decoding error probabilit...

Accurately determining dependency structure is critical to discovering a system’s causal organization. We recently showed that the transfer entropy fails in a key aspect of this—measuring information flow—due to its conflation of dyadic and polyadic relationships. We extend this observation to demonstrate that this is true of all such Shannon information measures when used to analyze multivaria...

Timed Shannon circuits have been proposed as a synthesis approach for a low power optimization technique at the logic level since overall circuit switching probabilities may be reduced. An improvement in the application of this principle for multi-output circuits is presented. Techniques that trade area for power reduction and a method for minimizing the overall circuit switching probability ar...