DeepAI
Log In Sign Up

Search-oriented Differentiable Product Quantization

04/16/2021
by   Shitao Xiao, et al.
7

Product quantization (PQ) is a popular approach for maximum inner product search (MIPS), which is widely used in ad-hoc retrieval. Recent studies propose differentiable PQ, where the embedding and quantization modules can be trained jointly. However, there is a lack of in-depth understanding of appropriate joint training objectives; and the improvements over non-differentiable baselines are not consistently positive in reality. In this work, we propose Search-oriented Product Quantization (SoPQ), where a novel training objective MCL is formulated. With the minimization of MCL, query and key's matching probability can be maximized for the differentiable PQ. Besides, VCS protocol is designed to facilitate the minimization of MCL, and SQL is leveraged to relax the dependency on labeled data. Extensive experiments on 4 real-world datasets validate the effectiveness of our proposed methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/27/2019

New Loss Functions for Fast Maximum Inner Product Search

Quantization based methods are popular for solving large scale maximum i...
11/12/2019

Norm-Explicit Quantization: Improving Vector Quantization for Maximum Inner Product Search

Vector quantization (VQ) techniques are widely used in similarity search...
03/25/2019

Local Orthogonal Decomposition for Maximum Inner Product Search

Inverted file and asymmetric distance computation (IVFADC) have been suc...
07/31/2022

Symmetry Regularization and Saturating Nonlinearity for Robust Quantization

Robust quantization improves the tolerance of networks for various imple...
08/02/2021

Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performance

Recently, Information Retrieval community has witnessed fast-paced advan...
04/01/2022

Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddings

Vector quantization (VQ) based ANN indexes, such as Inverted File System...
03/09/2022

Givens Coordinate Descent Methods for Rotation Matrix Learning in Trainable Embedding Indexes

Product quantization (PQ) coupled with a space rotation, is widely used ...