Disentangled Information Bottleneck

The information bottleneck (IB) method is a technique for extracting that relevant predicting the target random variable from source variable, which typically implemented by optimizing IB Lagrangian balances compression and prediction terms. However, hard to optimize, multiple trials tuning values of multiplier are required. Moreover, we show performance strictly decreases as gets stronger during Lagrangian. In this paper, implement perspective supervised disentangling. Specifically, introduce Disentangled Information Bottleneck (DisenIB) consistent on compressing maximally without loss (maximum compression). Theoretical experimental results demonstrate our maximum compression, performs well in terms generalization, robustness adversarial attack, out-of-distribution detection,