Flow-based Algorithms for Improving Clusters: A Unifying Framework, Software, and Performance

04/20/2020
by   K. Fountoulakis, et al.
0

Clustering points in a vector space or nodes in a graph is a ubiquitous primitive in statistical data analysis, and it is commonly used for exploratory data analysis. In practice, it is often of interest to “refine” or “improve” a given cluster that has been obtained by some other method. In this survey, we focus on principled algorithms for this cluster improvement problem. Many such cluster improvement algorithms are flow-based methods, by which we mean that operationally they require the solution of a sequence of maximum flow problems on a (typically implicitly) modified data graph. These cluster improvement algorithms are powerful, both in theory and in practice, but they have not been widely adopted for problems such as community detection, local graph clustering, semi-supervised learning, etc. Possible reasons for this are: the steep learning curve for these algorithms; the lack of efficient and easy to use software; and the lack of detailed numerical experiments on real-world data that demonstrate their usefulness. Our objective here is to address these issues. To do so, we guide the reader through the whole process of understanding how to implement and apply these powerful algorithms. We present a unifying fractional programming optimization framework that permits us to distill out in a simple way the crucial components of all these algorithms. It also makes apparent similarities and differences between related methods. Viewing these cluster improvement algorithms via a fractional programming framework suggests directions for future algorithm development. Finally, we develop efficient implementations of these algorithms in our LocalGraphClustering python package, and we perform extensive numerical experiments to demonstrate the performance of these methods on social networks and image-based data graphs.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 6

page 8

11/29/2018

Flow-Based Local Graph Clustering with Better Seed Set Inclusion

Flow-based methods for local graph clustering have received significant ...
08/31/2020

Structured Graph Learning for Clustering and Semi-supervised Classification

Graphs have become increasingly popular in modeling structures and inter...
10/22/2019

Hypergraph clustering with categorical edge labels

Graphs and networks are a standard model for describing data or systems ...
01/30/2018

Weighted Community Detection and Data Clustering Using Message Passing

Grouping objects into clusters based on similarities or weights between ...
08/17/2018

Semi-Supervised Cluster Extraction via a Compressive Sensing Approach

We use techniques from compressive sensing to design a local clustering ...
11/03/2017

Distributed Graph Clustering and Sparsification

Graph clustering is a fundamental computational problem with a number of...
06/12/2020

Distance-based phylogenetic inference from typing data: a unifying view

Typing methods are widely used in the surveillance of infectious disease...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.