Learning Sublinear-Time Indexing for Nearest Neighbor Search

01/24/2019
by   Yihe Dong, et al.
12

Most of the efficient sublinear-time indexing algorithms for the high-dimensional nearest neighbor search problem (NNS) are based on space partitions of the ambient space R^d. Inspired by recent theoretical work on NNS for general metric spaces [Andoni, Naor, Nikolov, Razenshteyn, Waingarten STOC 2018, FOCS 2018], we develop a new framework for constructing such partitions that reduces the problem to balanced graph partitioning followed by supervised classification. We instantiate this general approach with the KaHIP graph partitioner [Sanders, Schulz SEA 2013] and neural networks, respectively, to obtain a new partitioning procedure called Neural Locality-Sensitive Hashing (Neural LSH). On several standard benchmarks for NNS, our experiments show that the partitions found by Neural LSH consistently outperform partitions found by quantization- and tree-based methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

01/24/2019

Learning Space Partitions for Nearest Neighbor Search

Space partitions of R^d underlie a vast and important class of fast near...
12/22/2017

Lattice-based Locality Sensitive Hashing is Optimal

Locality sensitive hashing (LSH) was introduced by Indyk and Motwani (ST...
01/19/2017

Optimized Spatial Partitioning via Minimal Swarm Intelligence

Optimized spatial partitioning algorithms are the corner stone of many s...
05/13/2023

Indexing and Partitioning the Spatial Linear Model for Large Data Sets

We consider four main goals when fitting spatial linear models: 1) estim...
02/21/2023

Lightweight-Yet-Efficient: Revitalizing Ball-Tree for Point-to-Hyperplane Nearest Neighbor Search

Finding the nearest neighbor to a hyperplane (or Point-to-Hyperplane Nea...
08/11/2021

Learning to Hash Robustly, with Guarantees

The indexing algorithms for the high-dimensional nearest neighbor search...
06/16/2022

Unsupervised Space Partitioning for Nearest Neighbor Search

Approximate Nearest Neighbor Search (ANNS) in high dimensional spaces is...

Code Repositories

learn-to-hash

Neural LSH [ICLR 2020] - Using supervised learning to produce better space partitions for fast nearest neighbor search.


view repo

Please sign up or login with your details

Forgot password? Click here to reset