The Information Bottleneck and Geometric Clustering

The information bottleneck and geometric clustering

The information bottleneck (IB) approach to clustering takes a joint distribution P (X,Y ) and maps the data X to cluster labels T which retain maximal information about Y (Tishby et al., 1999). This objective results in an algorithm that clusters data points based upon the similarity of their conditional distributions P (Y | X). This is in contrast to classic “geometric clustering” algorithms ...

Geometric Clustering Using the Information Bottleneck Method

We argue that K–means and deterministic annealing algorithms for geometric clustering can be derived from the more general Information Bottleneck approach. If we cluster the identities of data points to preserve information about their location, the set of optimal solutions is massively degenerate. But if we treat the equations that define the optimal solution as an iterative algorithm, then a ...

Information Bottleneck Co-clustering

Co-clustering has emerged as an important approach for mining contingency data matrices. We present a novel approach to co-clustering based on the Information Bottleneck principle, called Information Bottleneck Co-clustering (IBCC), which supports both soft-partition and hardpartition co-clusterings, and leverages an annealing-style strategy to bypass local optima. Existing co-clustering method...

Conditional Information Bottleneck Clustering

We present an extension of the well-known information bottleneck framework, called conditional information bottleneck, which takes negative relevance information into account by maximizing a conditional mutual information score. This general approach can be utilized in a data mining context to extract relevant information that is at the same time novel relative to known properties or structures...

Fast Agglomerative Information Bottleneck Based Trajectory Clustering