Log In Sign Up

Attentive Clustering Processes

by   Ari Pakman, et al.

Amortized approaches to clustering have recently received renewed attention thanks to novel objective functions that exploit the expressiveness of deep learning models. In this work we revisit a recent proposal for fast amortized probabilistic clustering, the Clusterwise Clustering Process (CCP), which yields samples from the posterior distribution of cluster labels for sets of arbitrary size using only O(K) forward network evaluations, where K is an arbitrary number of clusters. While adequate in simple datasets, we show that the model can severely underfit complex datasets, and hypothesize that this limitation can be traced back to the implicit assumption that the probability of a point joining a cluster is equally sensitive to all the points available to join the same cluster. We propose an improved model, the Attentive Clustering Process (ACP), that selectively pays more attention to relevant points while preserving the invariance properties of the generative model. We illustrate the advantages of the new model in applications to spike-sorting in multi-electrode arrays and community discovery in networks. The latter case combines the ACP model with graph convolutional networks, and to our knowledge is the first deep learning model that handles an arbitrary number of communities.


page 8

page 15


Deep Amortized Clustering

We propose a deep amortized clustering (DAC), a neural architecture whic...

Microclustering: When the Cluster Sizes Grow Sublinearly with the Size of the Data Set

Most generative models for clustering implicitly assume that the number ...

The information bottleneck and geometric clustering

The information bottleneck (IB) approach to clustering takes a joint dis...

Probabilistic Clustering of Time-Evolving Distance Data

We present a novel probabilistic clustering model for objects that are r...

Attentive Representation Learning with Adversarial Training for Short Text Clustering

Short text clustering has far-reaching effects on semantic analysis, sho...

CRAFT: ClusteR-specific Assorted Feature selecTion

We present a framework for clustering with cluster-specific feature sele...

Multiobjective hBOA, Clustering, and Scalability

This paper describes a scalable algorithm for solving multiobjective dec...

Code Repositories


Implementation of the GCN-CCP algorithm.

view repo