Learning Sublinear-Time Indexing for Nearest Neighbor Search

by   Yihe Dong, et al.

Most of the efficient sublinear-time indexing algorithms for the high-dimensional nearest neighbor search problem (NNS) are based on space partitions of the ambient space R^d. Inspired by recent theoretical work on NNS for general metric spaces [Andoni, Naor, Nikolov, Razenshteyn, Waingarten STOC 2018, FOCS 2018], we develop a new framework for constructing such partitions that reduces the problem to balanced graph partitioning followed by supervised classification. We instantiate this general approach with the KaHIP graph partitioner [Sanders, Schulz SEA 2013] and neural networks, respectively, to obtain a new partitioning procedure called Neural Locality-Sensitive Hashing (Neural LSH). On several standard benchmarks for NNS, our experiments show that the partitions found by Neural LSH consistently outperform partitions found by quantization- and tree-based methods.


page 1

page 2

page 3

page 4


Learning Space Partitions for Nearest Neighbor Search

Space partitions of R^d underlie a vast and important class of fast near...

Lattice-based Locality Sensitive Hashing is Optimal

Locality sensitive hashing (LSH) was introduced by Indyk and Motwani (ST...

Optimized Spatial Partitioning via Minimal Swarm Intelligence

Optimized spatial partitioning algorithms are the corner stone of many s...

Indexing and Partitioning the Spatial Linear Model for Large Data Sets

We consider four main goals when fitting spatial linear models: 1) estim...

Lightweight-Yet-Efficient: Revitalizing Ball-Tree for Point-to-Hyperplane Nearest Neighbor Search

Finding the nearest neighbor to a hyperplane (or Point-to-Hyperplane Nea...

Learning to Hash Robustly, with Guarantees

The indexing algorithms for the high-dimensional nearest neighbor search...

Unsupervised Space Partitioning for Nearest Neighbor Search

Approximate Nearest Neighbor Search (ANNS) in high dimensional spaces is...

Code Repositories


Neural LSH [ICLR 2020] - Using supervised learning to produce better space partitions for fast nearest neighbor search.

view repo

Please sign up or login with your details

Forgot password? Click here to reset