Sketch and Scale: Geo-distributed tSNE and UMAP

11/11/2020
by   Viska Wei, et al.
0

Running machine learning analytics over geographically distributed datasets is a rapidly arising problem in the world of data management policies ensuring privacy and data security. Visualizing high dimensional data using tools such as t-distributed Stochastic Neighbor Embedding (tSNE) and Uniform Manifold Approximation and Projection (UMAP) became common practice for data scientists. Both tools scale poorly in time and memory. While recent optimizations showed successful handling of 10,000 data points, scaling beyond million points is still challenging. We introduce a novel framework: Sketch and Scale (SnS). It leverages a Count Sketch data structure to compress the data on the edge nodes, aggregates the reduced size sketches on the master node, and runs vanilla tSNE or UMAP on the summary, representing the densest areas, extracted from the aggregated sketch. We show this technique to be fully parallel, scale linearly in time, logarithmically in memory, and communication, making it possible to analyze datasets with many millions, potentially billions of data points, spread across several data centers around the globe. We demonstrate the power of our method on two mid-size datasets: cancer data with 52 million 35-band pixels from multiple images of tumor biopsies; and astrophysics data of 100 million stars with multi-color photometry from the Sloan Digital Sky Survey (SDSS).

READ FULL TEXT
04/27/2018

Buffered Count-Min Sketch on SSD: Theory and Experiments

Frequency estimation data structures such as the count-min sketch (CMS) ...
10/29/2020

Active Sampling Count Sketch (ASCS) for Online Sparse Estimation of a Trillion Scale Covariance Matrix

Estimating and storing the covariance (or correlation) matrix of high-di...
11/15/2018

Sketch based Reduced Memory Hough Transform

This paper proposes using sketch algorithms to represent the votes in Ho...
04/04/2018

SketchMate: Deep Hashing for Million-Scale Human Sketch Retrieval

We propose a deep hashing framework for sketch retrieval that, for the f...
04/23/2019

Block-distributed Gradient Boosted Trees

The Gradient Boosted Tree (GBT) algorithm is one of the most popular mac...
12/25/2017

Efficient Algorithms for t-distributed Stochastic Neighborhood Embedding

t-distributed Stochastic Neighborhood Embedding (t-SNE) is a method for ...
08/09/2022

Enabling Efficient and General Subpopulation Analytics in Multidimensional Data Streams

Today's large-scale services (e.g., video streaming platforms, data cent...