Log In Sign Up

Search-oriented Differentiable Product Quantization

by   Shitao Xiao, et al.

Product quantization (PQ) is a popular approach for maximum inner product search (MIPS), which is widely used in ad-hoc retrieval. Recent studies propose differentiable PQ, where the embedding and quantization modules can be trained jointly. However, there is a lack of in-depth understanding of appropriate joint training objectives; and the improvements over non-differentiable baselines are not consistently positive in reality. In this work, we propose Search-oriented Product Quantization (SoPQ), where a novel training objective MCL is formulated. With the minimization of MCL, query and key's matching probability can be maximized for the differentiable PQ. Besides, VCS protocol is designed to facilitate the minimization of MCL, and SQL is leveraged to relax the dependency on labeled data. Extensive experiments on 4 real-world datasets validate the effectiveness of our proposed methods.


page 1

page 2

page 3

page 4


New Loss Functions for Fast Maximum Inner Product Search

Quantization based methods are popular for solving large scale maximum i...

Norm-Explicit Quantization: Improving Vector Quantization for Maximum Inner Product Search

Vector quantization (VQ) techniques are widely used in similarity search...

Local Orthogonal Decomposition for Maximum Inner Product Search

Inverted file and asymmetric distance computation (IVFADC) have been suc...

Symmetry Regularization and Saturating Nonlinearity for Robust Quantization

Robust quantization improves the tolerance of networks for various imple...

Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performance

Recently, Information Retrieval community has witnessed fast-paced advan...

Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddings

Vector quantization (VQ) based ANN indexes, such as Inverted File System...

Givens Coordinate Descent Methods for Rotation Matrix Learning in Trainable Embedding Indexes

Product quantization (PQ) coupled with a space rotation, is widely used ...